THE DEVELOPMENT OF AERIAL TRIANGULATION TOWARDS LARGE BLOCKS

Dear professor Guoqing Zhou, I thank you very much for your kind invitation to attend this important meeting in Guilin. It is certainly a most important meeting, as the topic of "Big Data" is presently of high actuality. I must apologize that I cannot come to China to attend the meeting because my health condition is poor, it does not allow to travel large distances. Nevertheless, I submit a paper. And I am very glad that professor Deren Li will read it on my behalf. I am particularly grateful to him. He has been my best doctor student, and I still appreciate his doctor thesis. It was a most remarkable piece of research. And I believe that it has been the basis for his great career, which I have pursued with great interest. In any case I thank him very much that he has agreed to present my paper.

The decision to concentrate on computational methods for photogrammetry implied the development of software for block adjustments, from the beginning. We started with it in the early 1970 years, beginning with independent model block adjustment (PAT-M). After that came the true 3-dimensional bundle adjustment (PAT-B). The block sizes in those early years were not very large. They went hardly beyond blocks of 200 aerial photographs, and that remained for some time. However, our software could handle larger blocks. But its application depended mainly on large computers which, at that time, were only available at central university or government computing centers. Therefore the software installation had often to be modified in order to be adapted to certain computers.
For a few years photogrammetry was applied to cadastral surveys. The block sizes were not necessarily larger, but the number of unknown cadastral points to be measured went up to 10000 or much more per block. However, it did not cause additional software problems, because of the special structure of the normal equation matrix. The blocks remained within the range of a few hundred images at that time. But the private companies acquired more and more middle class computers, which vastly spread the application of block adjustment.
Occasionally blocks of up to 400 or 600 images became applied. I remember a block in South Africa which contained 2000 images. That was the largest block which I knew of in the 1970s.
During the 1980 years the Stuttgart software had been extended to more powerful performance. That implied the acceptance of much larger blocks, with arbitrary overlaps and arbitrary numbers of points, in combination with any consistent numbering system. It also included the automatic blunder detection, as well as the measurement of camera station positions by GPS and camera orientation data. Some time later digital cameras appeared on the market and our software had to be adapted to accept digital image data. Consequently, the automatic determination of digital tie-points or groups of tie-points by least squares image matching or feature based matching became applicable. Everything was combined in the MATCH-AT software.

THE BALTIMORE COUNTY PROJECT
The central part of my paper will deal with the Baltimore County project. It was a very large project which made full use of aerial and terrestrial GPS. Such a large system had not before been applied, as far as we knew. It was based on a remarkable decision in 1994 by the county authority which was run by director William Bond. He had in mind to establish a large scale general purpose geographic information system (GIS) for the whole county, based entirely on photogrammetry. The information system was successfully finished in the year 2000. It captured 31 types of objects. I will not deal here any further with the GIS project, but will concentrate on the block adjustments for which I was responsible. I was charged with the planning of the photogrammetric part, which was supposed to provide the geo-reference for the GIS. I decided to plan a big methodical jump by relying almost entirely on aerial GPS. It means that for each camera station the position was measured by aerial GPS and later used in the block adjustments. That had never before been applied for large blocks. But I was sure that it would work very well. The local photogrammetric consultant was Yogendra Singh, a former student of mine. I am most grateful to him as he followed my planning without any hesitation. He did not consult other experts. Most of them would have disagreed with my courageous planning, as no previous experience was available. The photo flights were flown in 1994, 1995, and 1996 with image scale 1:6000. The three blocks covered 125, 260, and 290 square miles (see Fig.  1-3). Which amounted to 731, 1612, and 1720 images, respectively, altogether about 4000 images. Their result was excellent. The _0 values of the first two blocks amounted to 4.6µm and 5.8 µm. The r.m.s. residual errors of the image points in x and y were 3.6µm and 3.2 µm for the first block and 4.5µm and 4.2µm, respectively, for the second. These values demonstrate the high accuracy of the aerial triangulations. It was in agreement with previous experience. Block 1, however, contained certain difficulties, as its eastern and southern sides followed the irregular coast line of the Chesapeake Bay. But the US National Oceanographic Administration had a software program by which the tidal movement of the water level in the Bay was modelled, as a function of location and time. Based on that model we could ask photogrammetrists to identify and measure many sea-shore points which would then be used as vertical control points. The r.m.s. value of 411 such sea-shore points amounted to 7.6 cm. That is certainly a most excellent value. It contained the photogrammetric measurement of the sea-shore points as well as the errors of the tidal modelling. I mention here the two first blocks of the project only briefly. Their photography was on color film and the block adjustments were treated by conventional analytical aerial triangulation. Their internal accuracy value has been mentioned above. And the absolute accuracy confirmed its high performance. The r.m.s. values assessed by 18 horizontal and 112 vertical checkpoints in block 1, respectively of 28 horizontal and 62 vertical check-point s in block 2 reached for block 1 in x and y 3.8 cm and 6.2 cm, and 9.4 cm in z. And for block 2 similar values were obtained, namely in x and y 8.2 cm and 6.8 cm, and 8.9 cm in z.
Let us now consider block 3. It deserves special attention as it was to be treated by digital aerial triangulation. However, there was not yet any digital camera on the market. Therefore we had to digitize the original color photographs. That was done with the Zeiss SCAI scanner which produced in a lengthy procedure digital images with 21µm pixel size. The scanning gave actually about 120 million pixels per image and more than 200 billion pixels for the whole block. It took more than a month by day and night. It constituted certainly a great challenge which had not been met up to that time. The data were delivered in 71 discs with 5 GB each. A total of 292 GB had to be handled. And we had to buy a 300 GB RAID storage equipment, in order to be able to store all digital image data. And it is obvious that we thus moved into the area of big data. The large magnitude of digital image data could not be used directly for the adjustment of block 3. Instead, our software had to select and transfer a limited number of tie-points or groups of tie-points. Finally we got 84 well distributed tie-points per image. Of them were 21364 triple tie-points, 5797 four-and five-fold points and 10884 six-fold tie points. Some tie-points connected up to 11 images. The block adjustment of 1720 images remained an enormous computational task, one which we had not previously dealt with. The adjustment had to handle 294000 observations, including the aerial GPS positions, and had 134040 unknowns. Such a case of block adjustment was not yet common at the time. But our software had worked very well. The results of the block adjustment with the MATCH-AT program turned out to be excellent. The _0 value was 4.8µm, which was most remarkable in view of the large pixel size of 21 µm. That was equivalent to 0.23 pixel size on the ground. And the r.m.s. values of the residual image coordinate errors were 3.3µm and 3.8µm in x and y, which was only 0.16 respectively 0.18 pixel size. The absolute accuracy of the adjusted block, as assessed by 35 horizontal and 74 vertical check -points was 6.8 cm, and 7.2 cm, and 9.2 cm in x, y and z. That was again an excellent result. We were certainly very happy with it, although I had more or less anticipated it in my planning. But all companies involved had performed an excellent job.

BIG DATA
All examples which are mentioned here, referred to the period of more than 20 years ago. At that time the case of big data was not yet a general topic in photogrammetry, as well as not in surveying. Therefore, I myself had not considered the case of big data as a task to go into after my retirement. However, I have collected a number of other big projects which will be mentioned hereafter.
Aerial triangulation is generally available and applied nowadays. It has certainly experienced additional progress. But very large blocks are not often applied. Yet blocks of 10000 or more images happen occasionally in countries with large desert areas.
A general topic in photogrammetry is the establishment of city models. However, it is not sufficient to take only vertical images for that purpose. It is also necessary to acquire images of oblique orientation, in order to catch the facades of buildings. There are now cameras on the market which take simultaneously 4 more oblique images, together with the vertical image. These are two with forward and backward oblique orientation, and two to the left and right. I know of a German company which has flown the city of Berlin with such a camera, collecting all together 47500 images. And at my former company INPHO, which now belongs to the US Trimble company, the software for aerial triangulation has consequently been in-creased to accept up to 50000 images. That is the largest development as far as aerial triangulation is concerned. But it only touches the case of big data, unless digital cameras would be used.
Let us know take a look on very large data bases. I restrict myself to the state of Baden-Württemberg, of which Stuttgart is the capital. However, it has only 600000 inhabitants which is certainly not very much in Chinese conditions. The state has an area of 35750 km 2 , of which 38% are forest areas. The company TopScan has flown the total state during 2002 to 2005 for establishing a digital terrain model (DTM), based on aerial laser scanning. The mutual distance of the laser points was about 80 cm. From them a DTM grid of 1 m mesh size has been interpolated. It gave about 35 billion grid points. Recently the laser equipment has been improved and measurement of laser points has become 2-10 times denser. That density can be used to identify break-lines.
The former state survey department of Baden-Württemberg, now called State Office for Geo-Information and Land Development, has established some big data bases. There is a cadastral data base which contains nearly 9 million property units and altogether 45 million cadastral points. There is also a data base for buildings which contain nearly 7 million buildings. These are certainly large data bases for a relatively small state. But we can refer to them as big data.
At the end I want to thank again professor Deren Li that he has accepted to read my paper. I appreciate his readiness very much and repeat that we all had been very proud of him. I was informed recently that he has also been involved in the Chinese space program which concentrates on the moon for the time being. I may guess that we will soon receive image data from the moon. The block of images may not be very large in terms of images, as it depends on the image scale. But it will cover large parts of the surface of the moon. With that expectation I congratulate the Chinese space program for its great attempt and I wish the best of success.
I also want to thank all members of my institute in Stuttgart, as well as of the INPHO company including a number of doctor students and of guest scientists, for having successfully completed our development of aerial triangulation before the year 2000. And I wish the institute great success for the further scientific development. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3/W10, 2020 International Conference on Geomatics in the Big Data Era (ICGBD), 15-17 November 2019, Guilin, Guangxi, China