A Comparison of Accuracy between Point Clouds from Convergent Images and Spherical

Laser scan technology has proved competitive over photogrammetric methods. The result of that method – an automatically generated point cloud enables prompt and effi cient object modeling. The photogrammetry responses with dense point clouds based on photos. The method has developed thanks to proper digital photo processing algorithms and using them for automatic photo adjustment. Now mostly SLR cameras, calibrated for stabilized image distance (usually for fi xed lens) are used for taking photos in close range photogrammetry. Relatively small sensor matrix size in comparison with the size of a photo taken with an analog measuring camera causes the increase in number of photos covering the captured object. To limit the number of photos, panoramic images taken from partial photos made from the same center of projection are used. Such spherical or cylindrical panorama covers much space, replacing the large format central projection. In the case of photogrammetric photos of large architectural objects (such as churches, palaces, and castles) the use of panoramic images [3–5] enables the signifi cant reduction of time, improves the measurement economics, simplifi es the calculation process and manual processing, as few programs enable the function of automatic processing from panoramas. That is why panoramas taken by special panoramic cameras or properly taken and connected photos from digital cameras are an alternative for classic convergent photos. The accuracy of photogrammetric measurement results is also aff ected by the degree of noise removal that is why one of the goals of this work was to test the effect of removal of distortion from component photos on the accuracy of point clouds generated at the later stage. However, our basic research goal was to analyze the quality and economics of creating point clouds based on convergent photos and spherical panoramas


Introduction
Laser scan technology has proved competitive over photogrammetric methods.The result of that method -an automatically generated point cloud enables prompt and effi cient object modeling.The photogrammetry responses with dense point clouds based on photos.The method has developed thanks to proper digital photo processing algorithms and using them for automatic photo adjustment.Now mostly SLR cameras, calibrated for stabilized image distance (usually for fi xed lens) are used for taking photos in close range photogrammetry.Relatively small sensor matrix size in comparison with the size of a photo taken with an analog measuring camera causes the increase in number of photos covering the captured object.To limit the number of photos, panoramic images taken from partial photos made from the same center of projection are used.Such spherical or cylindrical panorama covers much space, replacing the large format central projection.In the case of photogrammetric photos of large architectural objects (such as churches, palaces, and castles) the use of panoramic images [3][4][5] enables the signifi cant reduction of time, improves the measurement economics, simplifi es the calculation process and manual processing, as few programs enable the function of automatic processing from panoramas.That is why panoramas taken by special panoramic cameras or properly taken and connected photos from digital cameras are an alternative for classic convergent photos.
The accuracy of photogrammetric measurement results is also aff ected by the degree of noise removal that is why one of the goals of this work was to test the effect of removal of distortion from component photos on the accuracy of point clouds generated at the later stage.However, our basic research goal was to analyze the quality and economics of creating point clouds based on convergent photos and spherical panoramas

Modeling Based on Panoramas
Recent years have brought substantial progress of methods of photo processing and computerized vision technology.It enabled the automatic modeling of objects using photos made from both ground stations or UAVs.The applications that enable 3D modeling based on digital photos mainly use such algorithms as SfM (Structure from Motion), DIM (Dense Image Maching), which are generally available and do not require the author's approval to using or editing them [8].Apart from classic modeling methods based on photos the following methods are also used: shape from the shadow, shape from the texture or shape from the contours [11] Modeling based on photos is used in many applications.It has been used successfully to prepare architectonic works [9,12] and to prepare objects based on the processing of very large databases from various digital cameras [1].

Determining the Orientation for Groups of Convergent Photos and Panoramas
The spherical panorama orientation process may be manual or automatic.The fi rst method was described by Fangi and d'Annibale, who proposed solutions based on latitude-longitude type projections and coplanarity equation [3,4].The automatic measurement on panoramas was described i.a. by [2].Bazarett i and his co-authors propose a method based on SIFT and SURF operators and the fundamental matrix to detect common elements on panoramas.The automatic methods enable much faster image measurement for the block adjustment, however, they do not always provide full control over the measurement-calculation process.
Structure from Motion is a method commonly used to adjust convergent photos and enables the determination of their orientation parameters [10].To combine photos the image alignment algorithms based on features (SIFT, SURF, SUSAN, FAST) are used.On individual photos the sets of features are defi ned and then searched on other photos using the descriptors [13].

Current Tools for Determination of Point Cloud Based on Photos
The development and free use of SfM algorithms enabled the possibility of signifi cant development of software for photo-based point cloud generation in recent years.Many open-source applications were developed.Some of the most popular ones are Furukawa's and Ponce's PMVS2 [6] Bundler developed by Snavely [14] and VirtualSFM by Wu.Bundler and PMVS2 complement each other.The fi rst based on SfM and SIFT algorithm enables the adjustment of the photo block and the creation of sparse point cloud.The second, based on previously acquired adjustment results, enables the generation of a dense point cloud.VisualSFM is an application with improved methods used in Photo Tourism project by combining SIFT-GPU with a multicore adjustment method based on bundles.As Bundler, it enables the generation of photo group orientation.The results obtained in the application can be used to match photos in PMVS/CMVS [7].
More and more commercial applications are also developed, enabling the user an automatic photo processing, point cloud and 3D model generation.One of the best known applications is Agisoft PhotScan that uses SIFT algorithm and bundle method to adjust a photo group.The application also uses modules to create triangle grids and generate textures, enabling the end-to-end 3D object model generation [8].

Data Type and Place of Data Acquisition
As the test object for the experiment the building A0, The Main Hall of AGH University of Science and Technology in Cracow, was selected.Two types of photos were taken -convergent and component for spherical panoramas.Both types of data were acquired using the Canon EOS 5D camera.An additional equipment used during the work was a robotic camera mount GIGA PAN Epic Pro.
As the object of modeling in the test the most representative and also the most complex part of the Hall was selected -its southern wall, where prexy and dean lodges are located.18 convergent photos were taken using the camera with f = 24 mm focal length, which -depending on the distance to captured objects -enabled to get a terrain pixel GSD = 1.5-3 mm.
During the fi eld work some component photos were also taken to achieve four independent spherical panoramas, 70 per each one.To do this the focal length f = 35 mm was used, which enabled to get a terrain pixel GSD = 1-4 mm.
The technological processes of taking convergent and component photos are very diff erent.To make a proper panorama based on component photos it is required to use a special dedicated head.The head enables the minimization of the parallax error eff ect that appears when individual images do not have a common projection center.The time of photo taking is much shorter than in the case of component photos for panoramas, even in the case of non-automatized head [15].
Aside from the photo capturing the fi eld works also included signaling and the measurement of 25 photo-points and control points.The measurement with the polar method was made using the Trimble 5503 DR Total Station Series and the accuracy of coordinate determination of measured points was 3-5 mm.The axes of the coordinate system were assumed to be relatively parallel to the room walls and the X axis was the one that ran along the depth of the object.

Determining the Eff ect of Distortion Removal from Panoramic Component Photos
Spherical panoramas were created using the PTGui software.The authors claim that the application generates distortion-free panoramas.During the test this assumption was verifi ed by comparing the results of photogrammetric measurement from the panoramas (with distortion-free component photos) with the measurement made using the panoramas made of uncorrected photos.
Two programs were chosen to run the distortion removal process.The fi rst one was PTLens software and the second was Quadro developed by Dr Eng.Jakub Kolecki from the AGH University of Science and Technology.
PTLens is an application with embedded lens database with data related to distortion parameters.It resamples the photo on the basis of distortion of the lens, not a given piece.
Quadro software enables the user to defi ne radial and tangential distortion as well as the coordination of the principal point of the photo and image distance.As a result resampled, distortion-free images are obtained.However, it is necessary to calibrate the lens using a diff erent software.It was based on the photo bundle adjustment with self-calibration using the Agisoft PhotoScan software.Due to the fact that the distortion parameters are diff erently defi ned than in Quadro, they were re-calculated.
The accuracy analysis of photogrammetric measurement results was done for three data sets -point clouds generated from the panoramas, for which distortion was removed from the component photos and for which such a process was omitted.According to the authors' declaration, PTGui does it in the panorama creation process.
To obtain the panorama georeferences, the same four control points were used each time.The number of control points was reduced on purpose, so that the redundancy would not improve the model geometry.They were the basis for a preliminary accuracy analysis (Tab.1).To compare -the table also includes the results of adjusting the convergent photo group into the same control points.As can be seen from the comparison of mean deviation values on the panorama control points and convergent photos, the accuracy analysis based on the adjustment of models with four control points does not allow for an assumption that removing distortion from component photos before the panorama creation process increases the accuracy of the tested models.To verify the eff ect of distortion removal from the panorama component photos it was decided to verify the external accuracy of the colored point models.The coordinates on the control points were measured.Unfortunately out from 18 artifi cial and natural control points put onto the test wall, only 8-11 points were identifi able and measurable depending on the model.The results of the comparison of the photogrammetrically determined coordinates and from the geodetic measurement were shown in Table 2.It means that most probably the algorithm implemented in the PTGui software used to remove the aforementioned image geometric error is suffi cient and there is no need to do an additional photo correction [15].The root mean square of spatial deviations was determined from the formula: where: X t , Y t , Z t -coordinates of points determined by the geodetic method, X p , Y p , Z p -coordinates of point measured on the model, n -number of observations.
The accuracy of models based on panoramas, for which the distortion was removed from the component photos, is only slightly higher than in the case of the model based on the original panoramas.The diff erence may be the result of a real diff erence in model accuracy and of measurement errors of an observer who reads the control point location on the point model.The result of image matching in the PhotoScan software is a colored point cloud.It is not a continuous model in some places, due to the lower density of points in the generated cloud, precise determination of control point location was diffi cult, which resulted in worse accuracy of the measurement.It was noted that the greatest deviations were found on natural and non-signaled photo-points, located at the height above 4 m.In such places clouds were signifi cantly deformed due to worse conditions of the algorithm operation caused by the perspective, which was clearly visible on fl at objects -walls and columns of the Hall.

Comparison of Point Clouds from Panoramas and Convergent Photos
The main goal of the project was to compare the point models created on the basis of convergent photos and spherical panoramas.It included the quality, accuracy of point clouds and their time-consuming creation.
The fi rst criterion taken into consideration during the comparison was the time required for creating images ready for processing.It is much less time-consuming and troublesome to create panoramic images.Capturing component photos in the fi eld for 4 panoramas of 70 photos took about 50 minutes.It was also necessary to combine them and save as one image.This part of the work took 1 hour.It should be noted that the processes of combining and exporting were done using a quad-core CPU computer with 16 GB RAM memory.In the case of less powerful computers the process of combining and saving panoramas is much longer.Taking 18 convergent photos for later processing took about 5-10 minutes.
The time of point cloud generation and their density is another comparable factor.The process of grid alignment and model generation for 18 convergent photos with resolution of 4368 × 2912 pixels took about 2 hours.In the case of 4 spherical panoramas (15,000 × 7,500 pixels) the time was twice longer -it took about 4 hours.During both tests the same computer and the same internal sett ings of the PhotoScan software were used.During the result comparison, you should remember that in the case of panoramic photos whole spheres were processed, which signifi cantly aff ected the processing time.Agisoft software enables the selection of image fragments to prepare a model, however, the trials connected with this option did not give satisfactory results, so it was decided to work on full images.The fi nal results were two point clouds.In order to compare their resolution a rectangular prism was defi ned that limited the tested point models.The number of cloud points generated on the basis of panoramic images was about 724 000 and the cloud points from the convergent photos included approx.6 900 000.
The last compared element was the quality of the generated models.The accuracy analysis results for the point clouds from the original panoramas and convergent photos were shown in Table 2.You can see a signifi cant diff erence in model quality.The RMSE value determined in control points for the cloud of convergent photos is thrice lower than the value of panoramic clouds.Moreover, regardless of the control point type (artifi cial or natural) the values of deviations are similar and do not exceed 35 mm.As this may be a result of a more dense point cloud taken from convergent photos, which results in bett er identifi cation of control points on the point model and a lower error of determining the spatial coordinates, it was decided to verify the eff ect of photo type (convergent, panoramas) not on the fi nal result -the point cloud, but on the on intermediate stage -the photo grid.To do this, manually measured control points were added to the points and the coded control points automatically measured by software.Their spatial coordinates were achieved in the orientation result protocol.It turned out that the external accuracy of the models for panoramic photos was 36 mm and for convergent photos 20 mm regardless of the distortion correction.
Fitt ing the same panorama grids (distortion removed by PTGui) and convergent photos into 16 control points provided similar deviation results on photo points -for panoramas: RMSE ΔX = 5 mm, RMSE ΔY = 7 mm, RMSE ΔZ = 10 mm, RMSE ΔXYZ = 13 mm, for convergent photos: RMS ΔX = 10 mm, RMS ΔY = 5 mm, RMS ΔZ = 3 mm, RMSE ΔXYZ = 12 mm, and on control points the deviations calculated from panorama grids are still larger than from convergent photo grids.However, due to statistically insuffi cient trial of those points within the experiment done in this project, the results were not provided.
Much less accuracy of photogrammetric measurement from panoramic photos is caused by a few factors.The basic factor are the image errors caused by imperfection of spherical panorama made of component photos, whose centers of projection do not overlap due to determination of no-parallax point of the panoramic head (this error depends on us) and the eccentricity of the head rotation (this error depends on the manufacturer of the panoramic head).The panoramic image errors also include errors of combining component photos into a panorama by the PTGui software.

Conclusions
The experiments have shown that the initial removal of distortion from the panorama component photos has no infl uence on the increase of point model accuracy.Slight improvement of the model accuracy and time-consuming generation process confi rm the thesis that there is no need to correct the component photo distortion before combining them into panoramas.It was also noted that generating point clouds based on convergent photos is more eff ective and enables more accurate 3D models.The time is much shorter and they do not require additional processing, combination or correction.On their basis it is possible to achieve a very dense point cloud with high geometric accuracy.To solve the problem it is required to run additional tests on more panoramas, close to the number of convergent photos used in the experiment.More panoramic images would increase the number of tie points for individual points and therefore would increase the number of cloud points.It is also possible it would aff ect the accuracy of the geometric model.The popularity of panoramas for photogrammetric measurements was caused by the manual modeling.Now, the software that enables automatic modeling does not require a decrease in the number of images, so using panoramas is less reasonable.
Due to hardware limitations, creating point clouds from photos has been mainly used in aerial photogrammetry.Progress in technology and software enables using this technique in close range photogrammetry.The possibility of using inexpensive digital cameras (in relation to laser scanners) provides many occasions to develop this method and to increase the competition with ground scanning.It is therefore important to optimize the process of photo capturing, creating new algorithms of automatic point cloud and 3D model generation and their implementation in the photogrammetric software.

Table 1 .
The accuracy of the adjustment the models from the panoramas with photo-points

Table 2 .
The external accuracy of models from panoramic and convergent photos