Combined Bundle Adjustment with Distances and GPS Observations * * 1

Today, automatic image orientation and automatic image-based dense reconstruction algorithms are often implemented in professional photogrammetric software. At the same time progress in computer vision brings many advantages to everyone who is dealing with image-based object modeling. A number of free, often on-line software for automatic sparse or dense reconstruction have appeared during the last years [8]. The majority of them allow only for fully automatic reconstruction [1, 7] which is good for amateurs but bad for specialists, as controlling some important aspects of processing, like camera calibration is not available. As a result generated models are out of scale and proper angular orientation, and cannot be used as metric products. Addressed problems have been overcome in professional photogrammetric software where the bundle adjustment with control points is a standard. However, control points collection is a time consuming process even for small objects. Besides it involves using additional survey equipment. As an alternative approach the direct geo-referencing methods seem to be a very att ractive solution. Nowadays a number of cameras are equipped with integrated GPS, or have the interface open for an external receiver. Smartphones and tablets as a standard are equipped with GPS, MEMS inertial sensors and cameras which still improve their resolution and image quality. Low-cost devices providing image direct geo-referencing could be potentially utilized to improve the accuracy of obtained photogrammetric products, bringing new functionality to free, image-based modelling software. The only problem with low-cost GPS receivers is the low accuracy of positioning. The positioning errors can be in the considerable extend compensated by the photogrammetric observations [2, 5] however not completely. GPS measurements of projection centers, when conducted in the network with good controllability, allow


Introduction
Today, automatic image orientation and automatic image-based dense reconstruction algorithms are often implemented in professional photogrammetric software.At the same time progress in computer vision brings many advantages to everyone who is dealing with image-based object modeling.A number of free, often on-line software for automatic sparse or dense reconstruction have appeared during the last years [8].The majority of them allow only for fully automatic reconstruction [1,7] which is good for amateurs but bad for specialists, as controlling some important aspects of processing, like camera calibration is not available.As a result generated models are out of scale and proper angular orientation, and cannot be used as metric products.Addressed problems have been overcome in professional photogrammetric software where the bundle adjustment with control points is a standard.However, control points collection is a time consuming process even for small objects.Besides it involves using additional survey equipment.As an alternative approach the direct geo-referencing methods seem to be a very att ractive solution.
Nowadays a number of cameras are equipped with integrated GPS, or have the interface open for an external receiver.Smartphones and tablets as a standard are equipped with GPS, MEMS inertial sensors and cameras which still improve their resolution and image quality.Low-cost devices providing image direct geo-referencing could be potentially utilized to improve the accuracy of obtained photogrammetric products, bringing new functionality to free, image-based modelling software.The only problem with low-cost GPS receivers is the low accuracy of positioning.The positioning errors can be in the considerable extend compensated by the photogrammetric observations [2,5] however not completely.GPS measurements of projection centers, when conducted in the network with good controllability, allow for the estimation of angular elements of external orientation using only low cost inertial sensors [6] or even without using inertial sensors at all [2].Looking for cheap and simple approaches to improving accuracy in small-area combined image networks one may think about direct distance measurements.Ellum [3] tried to include single distance observations into a small terrestrial image network achieving some improvement in fi nal accuracies.Kolecki [5], similarly to Ellum, didn't notice a considerable decrease of errors.However, both authors used survey-grade GNSS receivers in their systems.Kolecki [5] in his simulations proves that distance observations have the potential to improve accuracy of image orientation only if lower accuracy GPS is involved.In the case of using a survey-grade receiver, distance measurements are simply redundant.In this paper more experiments and simulations are addressed to give deep insight into the nature of distance observations and their impact on adjustment results.

Functional Model
The functional model of applied bundle adjustment consists of three kinds of equations.The majority of observations are the image coordinates, bringing into the model equations of collinearity (1) in its usual form.The observed external orientation parameters are included using equations ( 2) and (3).The distance observations are included by adding into the model equation ( 4): where: 0 k X , 0 k Y and 0 k Z stand for projection center coordinates, X i , Y i i Z i are the object point coordinates, α, ν and κ are the angles of image orientation, the r coeffi cients come from the rotation matrix, ck, x 0 and y 0 are the internal orientation parameters.As given in the equation ( 4), the distances are always measured between object points that must also be marked in the images as tie points.

Data Preparation
The aim of this experiment is the evaluation of adjustment accuracy with and without distances in 'ideal conditions'.This means that the true external orientation parameters and their accuracy are known.Besides the distance observations that are free of errors can be provided.However it should be noticed that the infl uence of distances may depend on the overall network confi guration and the distance spatial distribution.The examined network is shown in Figure 1.It has a closed-loop confi guration to provide good controllability of observations.During the preparatory works this network was adjusted with 10 control points, to calculate the "truth" external orientation parameters.Then the 11 distances were calculated using the adjusted coordinates of tie points.As it can be seen in Figure 1 these distances are uniformly distributed.However it should be clear that some of them may be hard to determine by a fi eld survey.The adjusted projection centers (PC) coordinates can be treated as true values.In this experiment these values are corrupted by the zero-mean random Gaussian noise with the σ parameter changing from 5 to 300 mm in the intervals of 5 mm.To improve the reliability of the simulations 40 sequences of such a random number were generated for each projection center coordinate.This means that the network can be adjusted with the corrupted coordinates 40 times for each σ.The number of 40 seems to be a reasonable compromise between calculation time and the reliability of the results.Altogether, for all values of σ, there are 2,400 adjustments in each adjustment scenario.For each adjustment the standard deviations for check points are calculated and averaged within bins of 40.One should expect the averaged series of errors will form the ascending line in the σ over RMSE plot.

Simulation Scenarios
Having the data prepared, the simulations can be started.The following scenarios of adjustment are going to be carried out: -A0: 2,400 adjustments without distance observations and with constant a priori error of projection center coordinates, -B0: 2,400 adjustments without distance observations, with a priori errors of projection centers changing according to values of sigma in the random number series, -A1: 2,400 adjustments with only one distance and constant a priori error of projection center coordinates, -B1: 2,400 adjustments with only one distance and a priori errors consistent with the values of sigma.
-A11: 2,400 adjustments with 11 distances and constant a priori error of projection center coordinates, -B11: 2,400 adjustments with 11 distances and a priori errors consistent with the values of sigma.
In scenarios A the a priori errors of PC coordinates are 4 cm, 4 cm and 5 cm respectively for X, Y and Z.The aforementioned values seem to be the reasonable estimation of the expected accuracy of the real-time kinematic GPS survey in the built-up areas.The highest possible value of σ in the B scenarios (i.e. 30 mm) corresponds to the accuracy that may be achieved using pos-processed, code-only GPS surveys provided the good satellite visibility.

Accuracy
Plots showing the relative accuracies of bundle adjustment for all adjustment scenarios are presented in Figure 2 (scenarios A) and Figure 3 (scenarios B).It can be noticed (Fig. 2) that underestimation of a priori errors of geo-referencing results in systematic accuracy drop.Including distances helps to get slightly bett er results.However, the eff ect of using 11 distances instead of 1 seems to be surprisingly insignifi cant.To make the overall conclusion one can examine the 3D position error (Fig. 2d) and notice that including distances provides about 20-25% accuracy improvement.However it is clear that this rule may not hold in networks with diff erent image number and diff erent geometry.In practice such considerable under (or over) estimation of a priori errors is not likely to happen, but the experiment shows some general phenomena, and illustrates the importance of assessment of observations accuracy in the context of adjustment.The extent in which a priori accuracy is over estimated is illustrated in Figure 4, where the change of standard deviation of an observation of unit weight is shown.
When the weighs in adjustment process are consistent with the true accuracies of PC coordinates, the distance observations stop the check point errors from increasing (Fig. 3).Some remaining errors are still present (about 3 mm for each coordinate) possibly due to initial inaccuracy of check points, nevertheless the accuracy seems not to decrease, even though the PC errors reach 30 cm.However if the distances were excluded from the adjustment, the check point errors would increase noticeably (black lines in Figure 3c, d).What is more interesting, the number of observed distances has no infl uence on accuracy.The blue and red lines in the plots shown in Figure 3 almost coincide.It seems that in the assumed conditions of simulation of scenario B (true EO and true distances consistent with the observations, true a priori accuracies are known), observed distances do not help to correct the network geometry locally.Instead the distance observations correct the scale of the network, which in the case of no distances, would be disturbed by the erroneous observed PC coordinates.The rotation of the network may not be disturbed sufficiently to cause signifi cant check point errors.In scenario A the value of the standard deviation of unit weight is not infl uenced by increasing the errors of the observed PC coordinates (Fig. 4).However its value is lower than 1.0 due to the overestimated errors of the image coordinates of the tie points.

Scale
Further calculations were carried out in order to show the role the observed distance plays in the bundle adjustment of the test network, especially its infl uence on the scale.For scenarios B0 and B1 the 3D 7 degrees-of-freedom transformation was calculated according to the Horn closed-form solution [4].Scale checking was conducted for 15 networks in scenario B0 and for 15 networks in scenario B1.For each scenario checked networks were grouped in 3 bins, each containing 5 networks with the same errors of PC coordinates.The values of the PC coordinate error and corresponding average scale is given in Table 1.According to the terminology of Horn, as the right (target) data set the true check point coordinates were utilized.For scenario B1 the deviation of the scale from the unity is kept close to the level of 10 −4 irrespective of PC error.If the distance observation is removed (scenario B0) this deviation increases reaching the level of 10 −3 for the 15 cm error of PC coordinates and 10 −2 for the 30 cm error.Provided results show that using the distance observation can help to fi x the scale of the network, which may be disturbed to a large extend by erroneous direct geo-referencing.In Figure 5a it can be observed that the percentage of observation that is utilized in adjustment (complement of redundancy to the value of 1.0), in the scenario B1 increases from 10% to 99%.It means that for large PC coordinate errors, almost 100% of this single distance observation is utilized in the adjustment and the proper scaling of the network is almost fully dependent on the accuracy of this measurement.In the Figure 5b the mean redundancy has values closer to unity, as the total distance redundancy is distributed to single observations.The controllability of each distance is therefore bett er.What is quite interesting, in Figure 5a and in Figure 5b the blue line crosses the red line around the abscissa 40 mm -the value of PC X and Y coordinate error assumed for scenarios A.

Conclusions
The experiments presented in this paper show some phenomena that occur in the adjustment of distance-supported photogrammetric networks with direct geo-referenced images.Presented simulations allow for examining the interaction that takes place between observed projection center coordinates and distances for the image network with favorable geometrical confi guration (closed loop) and good controlability.In that case the distance observations are error-free, they can play a great role in stabilizing the scale of network, that in case of erroneous geo-referencing can be easily disturbed.However one very important aspect should be kept in mind, namely the proper weighting of the observed PC coordinates (and potentially angles).Underestimation of a priori errors may limit the corrective role of the distances and increase adjustment errors.Proper weighting of certain groups of observations may be achieved using the methods proposed by Ziobro [9,10] In practice photogrammetric networks may not have as good controlability as network presented in this paper (Fig. 1).Besides the true accuracy of PC coordinates remains unknown and the accuracy of measured distances is limited.Often distances cannot be measured in versatile directions like those in scenario B11.Similar tests, also for networks consisted of strips, are presented by Kolecki [5].However, in those simulations the true PC coordinate errors were unknown and the distances were measured between terrain points with limited accuracy.Under those conditions distances also helped to improve accuracy, however not in such a large extent as in the presented tests.
Photogrammetric processing, especially the close range measurements, can be supported by additional information about objects.This may include not only distances but also tie lines and imposed constraints like vertical, horizontal, parallel and orthogonal directions.Also, measurements with geo-referencing sensors should help improve the accuracy of created models opening fully-automated photogrammetry to many applications.

Fig. 1 .
Fig. 1.Test image network with camera stations and distances.Tie and check points are marked in black and red respectively

Fig. 2 .
Fig. 2. Relative accuracy of bundle adjustment with constant errors of PC coordinates in scenario A; black: no distances, red: one distance, blue: eleven distances

Fig. 3 .
Fig. 3. Relative accuracy of bundle adjustment with truth a priori errors of PC coordinates in scenario B; black: no distances, red: one distance, blue: eleven distances

Fig. 4 .Figure 5
Fig. 4. Dependency of the standard deviation of an observation of unit weight from the true error of observed PC coordinates; red -fi xed a priori errors of PC coordinates, blue -weights of observed PC coordinates consistent with true errors

Fig. 5 .
Fig. 5. Dependency of mean observed distance redundancy from the true error of observed PC coordinates: a) scenarios with one distance (A1, B1); b) scenario with 11 distances (A11, B11); red -fi xed a priori errors of PC coordinates, blue -weights of observed PC coordinates consistent with true errors

Table 1 .
Results of scale parameter estimation for networks with three diff erent error levels of PC coordinates adjusted within scenarios B0 and B1