MEASUREMENT ERROR WITH DIFFERENT COMPUTER VISION TECHNIQUES

The goal of this work is to offer a comparative of measurement error for different computer vision techniques for 3D reconstruction and allow a metrological discrimination based on our evaluation results. The present work implements four 3D reconstruction techniques: passive stereoscopy, active stereoscopy, shape from contour and fringe profilometry to find the measurement error and its uncertainty using different gauges. We measured several dimensional and geometric known standards. We compared the results for the techniques, average errors, standard deviations, and uncertainties obtaining a guide to identify the tolerances that each technique can achieve and choose the best.


INTRODUCTION
Several imaging techniques for accurate 3D reconstruction and their quality is improving rapidly.Some authors like (Hocken, 2011), (Maier-Hein, 2013), (Rashmita, 2013) and (Ng, 2011) believe that an evaluation is necessary for these techniques, so they assess the process algorithms to determine the quality of the measurements.
The purpose of this work is to assess the measurement error of passive stereoscopy, active stereoscopy, shape from contour and fringe profilometry using different known dimensional gauges.The techniques considered apply algorithms to fit a set of points from a gauge image to a particular geometry.The gauge measurement error define the quality of the technique.Authors as (Rashmita, 2013), (Ng, 2011) and (Salvi, 2007) suggest that industry can use these techniques for quality control if the measurement error and its uncertainty is under the verified tolerances.
The gauge measurement error is defined as the measure calculated with the technique minus the known value of the gauge.The nominal used lengths were 3 mm, 50 mm and 100 mm block gauges, a 10 mm metallic pin diameter, and a height of 15.081 mm of fine-grained 3D printed piece of gypsum, see Figure 1.The closest work to our own investigations has been proposed by (Ramos,2011), who instead of implemented techniques used different instruments with digitization techniques incorporated into * Corresponding author its measurement software and similar to our work he used different reference gauges.Most of the reported vision techniques in (Hocken, 2011), (Maier-Hein, 2013), (Rashmita, 2013), (Ng, 2011) and (Salvi, 2007) offer measurement errors using objects with no traceability.Our work report measurement errors using gauges with traceability to national standards.
This paper is organized as follows.Section 2 deals with the uncertainty applied to the results.In section 3, we implemented the 3D reconstruction techniques including references, calibration process, the particular experimental setup and the results.
The comparative results between techniques are given in section 4 and section 5 deals with the conclusions and discussion about results.

Uncertainty applied on results
In accordance with (ISO 14253-1, 1998) every measurement process must take into account the accuracy (ISO 14253-1, 1198) represented by a resultant error and its corresponding uncertainty.
We based the uncertainty calculation on the guide to the expression of uncertainty in measurement (GUM, 2010) 2011).The expanded uncertainty expression U for our error measurand is: where k is the coverage factor with a conventional value of two to achieve a probability of 94.5% of confidence in the calculated uncertainty.uc(y) is the combined uncertainty of each contributor y. u cal is the standard calibration uncertainty reported in the certificate of calibration of the gauge reference used.up is the standard uncertainty associated with the measurement process, it includes sources of uncertainty as repeatability, the measurement environment, subjections, measurement strategy and procedure, speed of measurement, and geometry of the measuring system.u b is the standard uncertainty associated with the systematic error b of the measurement process evaluated using the calibrated work piece, in our case the systematic error is the measurand so u b will not be considered.uw, is the standard uncertainty of the material and the variations of manufacture, it includes sources of uncertainty as variations in thermal expansion, errors of shape, roughness, elasticity, and plasticity.
Due to the error ranges of the implemented techniques and the quality of the gauges u cal , u b and uw are considered zero so Equation 1 is simplified as follows: From Equation 2, the uncertainty up is Equation3: where σx the standard deviation of the measurements defined in Equation4, and n is the number of measurements.
where n is the number of measurements, qj is the j measurement and q is the mean of the measurements.
All the techniques were implemented in a laboratory with a temperature of 20 o ± 3 o C , with artificial illumination and the absence of direct sunlight.The number of measurements made with each technique is at least n = 3 to calculate uncertainty.The uncertainties of calibration certificate for the gauges used are in the range of ±12 nm to ±250 nm which, as mention before, were consider as approximately zero for the case of the implemented techniques.

Passive Stereoscopy Technique (PST)
According with (Szeliski, 2010), in computer vision, the topic of Passive Stereoscopy Technique (PST), specially stereo matching, has been one of the most widely studied and fundamental problems, and continues to be one of the most active research areas.While photogrammetric matching concentrated mainly on aerial imagery, computer vision applications include modeling the human visual system, robotic navigation and manipulation, as well as view interpolation and image-based rendering, 3D model building, and mixing live action with computer-generated imagery.

Calibration method and model for PST
To calibrate the cameras we considered the model proposed by (Hall, 1982) and (Tsai, 1987) both uses the pinhole camera model.This model is extended with some correction parameters, such as radial lens distortion and tangential lens distortions that causes radial and tangential displacement in the image plane.
Camera calibration means the process of determining the internal camera geometric and optical characteristics (intrinsic parameters) and/or the 3D position and orientation of the camera frame relative to a certain world coordinate system (extrinsic parameters).The purpose of the calibration is to set a relationship between 3D points from the world and 2D points from the image, seen by the computer (Tsai, 1987), (Heikkila, 1997), and (Vilaca, 2010).
We performed cameras calibration for PST with the toolbox from (Bouguet, 2013) that uses a checkerboard calibration pattern.Table 1 shows the results of the intrinsic parameters for each camera and its uncertainties.The resulting 3D reconstruction depends on image quality.In order to obtain images of sufficient quality we acquired experimental images under good illumination conditions as mentioned in (Belhaoua, 2010) and (Muquit, 2006).
We fitted a plane π : ax + by + cz + d = 0 and a point p1 = (x1, y1, z1) outside the plane to calculate the measurements.The shortest distance from p1 to plane is defined in Equation5: where a, b, c and d are the coefficients of the plane and (x1, y1, z1) represents a point outside the plane.

Results of PST
We tested PST using 3 mm, 50 mm and 100 mm gauge blocks and the fine-grained 3D printed piece  of gypsum (Pyramid).Based on (Hocken, 2011), the main factors influencing the accuracy of a stereovision system are as follows: defects of optical systems, errors caused by image processing, quality of the target, arrangement of the cameras, calibration errors, thermal error and stability camera mounting.The reported errors in Table 2 occurred due to different factors: • Manual selection of correspondence points detected in both cameras, see Figure 2.
• The quality projection of the points is low.
• The angle arrangement of the cameras to the normal of the plane is less than 45 o .The technique shows measurement errors greater than 2 mm when measured dimension exceeds 50 mm which is higher than the reported in the literature (Belhaoua, 2010).As a result, this technique seems to be unsatisfactory for measuring dimensions over 50 mm.Table 2 shows that the increasing size of the gauge is directly proportional to the average error.

Active Stereoscopic Technique (AST).
One of the most popular active illumination devices is a laser or light stripe, which sweeps a plane of light across the scene or object while observing it from an offset viewpoint (Rioux, 1993) and (Curless, 1996), see Figure 3.The laser and the offset viewpoint (commonly a camera) enclose the Active Stereoscopic Technique (AST).As the stripe falls across the object, it deforms according to the shape of the surface it is illuminating.Then using optical triangulation we estimate the 3D locations of all the points seen in a particular stripe.
In more detail, knowledge of the 3D plane equation of the light stripe allows us to infer the 3D location corresponding to each illuminated pixel.The accuracy of light striping techniques improves by finding the exact temporal peak in illumination for each pixel (Curless, 1996).

Calibration method and model for AST
As in PST, the employed model is the pinhole, just the way to calculate extrinsic parameters is different.Table 1 shows right camera intrinsic parameters and its uncertainties.For this method is necessary to obtain the equation of the line-plane from the laser beam depicted in Figure 3.This work used four heights: 3 mm, 50 mm, and 100 mm gauge blocks, and the reference plane.We fit four points from the top of these gauge blocks and 400 points from the reference plane outside the gauge blocks, giving a total of 404 points used to determine the plane of the line.In total, fifteen planes must be found equivalent to firteen lines according to the pattern of light generated by the laser (see Figure 4(a)).However we only need to find the plane equation of the line that the experimental setup from Figure 4(a) allows passing.

AST experimental setup
This plane will lead to the dimensional reconstruction of gauges blocks, and allows the determination of the depth of the object from the camera, see Figure 3.

Results of AST
We used 3 mm, 50 mm and 100 mm length gauge blocks to test AST.Table 3 shows results calculated using fitted lines adjusted to the reference plane and a maximum point found on the peak of the signal see Figure 5.
According to     5. Furthermore, the surface material where the light beam is projected is a factor that contributes to noise.Table 3 also shows the average error for the 50 mm and 100 mm gauges blocks increases from 0.232 mm to 1.848 mm proportional to its size, it is proposed to use a correction factor (usually linear) that change with the dimensions of the piece to be measured.This technique consists on a triangulation principle between the laser transmitter, the measured point and the camera.(Hocken, 2011) mentioned that the measured results of this technique depend heavily on the structure and angle of inclination of the surface (see Figure 3).This leads to relatively large measuring uncertainties making the use of this technique suitable only for less demanding applications.(Hocken, 2011) mentioned that better results can  be achieved with laser sensors working according to the Focault principle.
3.3 Shape from Contour Technique (SCT) (Baumgart, 1974) introduced the idea of using contours for 3D reconstruction.Afterwards (Szeliski, 1993) proposed a noninvasive digitizer built using a rotary table and a single camera with the reconstruction procedure of Shape from Silhouette (SFS) or Shape from Contour (SFC).SFS has become one of the most popular methods for the reconstruction of static objects (Gonzalez-Barbosa, 2015), (Fitzgibbon, 1998) and (Fang, 2003).SFC presents results with errors for objects of high complexity or detailed (Rodríguez, 2008).

Calibration method and model for SCT
We based on (Gonzalez-Barbosa, 2015).The orthographic geometry illustrated in Figure 6, is the projection of a three dimensional point Q(X, Y, Z) over a plane image generates a point q(u, v) .Where u = X and v = Y .Then, the model Equation 6of orthographic projection in homogeneous coordinates where K is defined as the matrix of intrinsic parameters, and E as the matrix of extrinsic parameters, see Equation7 The elements of K include: α and β, the scale factors in image u and v axes, and γ the parameter describing the skewness of the two image axes.The matrix E represents the rigid transformation that carries the world coordinate system to the image coordinate system, composed of a rotation R ∈ R 3x3 and a translation T ∈ R 3 .
The calibration is carried out using the technique shown in (Fitzgibbon, 1998).The matrix of intrinsic parameters is denoted by: The Equation used to convert the data from 2D image coordinates to 3D world coordinates is defined by: where Q is a 3x1 vector representing 3D world coordinates and q is a 3x1 vector of each pixel from the subpixelic contour.The Shape reconstruction From the contour technique was tested using 3 mm, 50 mm gauge blocks and the 10 mm cylinder gauge.

SCT experimental setup
The SFC system used a telecentric lens (OPE TC 1364 by Opto Engineering), a turntable with a high accuracy encoder and 2.16 arc sec resolution (CR1-Z7 by Thorlabs), and calibration pattern (PT 036-0596 by Opto Engineering), depicted in Figure 7. Images of a rotating object illuminated from its back side are rendering.Consequently, the acquisition system can only see the contour of the object, as shown in Figure 8.We took 73 images with 5 degrees steps, i = 0, 5, . . ., 360.After the image acquisition, we applied a sub pixel extractor.The angle of rotation was calculated as the product of the image index with the rotation angle indicated by the turntable.
Using only one axis of rotation in all the images leads to inaccurate results.This inaccuracy is due to slight changes of the axis of rotation coming from illumination fluctuations and misalignment of the work piece with respect to the rotation of the turntable and the position of the camera.A dynamic rotation axis is necessary to reduce this inaccuracy.The first degree polynomial is a straight line fitted to all points over the subpixel contour image.This Equation is of the form: where u and v are pixel coordinates and A, B, C are the line coefficients.
Once calculated the dynamic axis of rotation, we have to perform a translation and rotation (eccentricity and leveling of the work piece with respect to the camera) of the coordinate system in pixel coordinates to the dynamic axis of rotation.This axis of rotation allows the 3D reconstruction with the accuracy showed in Table 4.At the same time, each one of the contour sub-pixel images was rotated around the X axis, and the gauge was plotted to show in real time the 3D reconstruction of the gauge, see Figure 9. Finally, the pixel coordinates were transformed to world coordinates using Equation 8.

Results of SCT
In this technique we measured gauges three times.The measurement system acquired 73 images every 5 o degrees in 360 o , equivalent to one and half hour of capture.In Table 4 we show an average error of -0.215 mm in the cylinder of 10 mm diameter, this is due to the blurred contour of the gauge coming from reflections over cylinder metallic surface (see Figure 8 affecting the measurement uncertainty, conversely well-defined contours for the ceramic gauge blocks.According to (Hocken, 2011), this technique is suitable to measure flat objects.On the other hand, where wide edges are involved, the interrelationship among illumination, the test object, and the imaging beam path must be taken into consideration in the uncertainty calculation.Important elements that contributed to the low uncertainty and repeatability of this technique were telecentric lens, turntable, and calibration pattern.Despite errors in images processing caused by surface structures, pits, and contamination the technique provided better repeatability and lower uncertainty than AST and PST.
We choose the temporal unwrapping profilometry technique tested in (Huntley, 1997) and (Xiao, 2010) because it is immune to noise and big gradients in test objects.It consists on projecting fringes with different periods at shifted phase; in our work we use, four images Ii shifted 90 degrees at t = 1, 2, . . ., 7 different frequencies.
The wrapped phase map for the value of t = 1 is calculated with the Equation 10 for phase shifting.
For the next phase maps, we use the Equation 11∆Φw where Finally, to find the unwrapped phase U, we apply the recursive equation with two consecutive wrapped phase maps.Equation 13 With the value of the unwrapped phase, we can perform the calibration to calculate the metric values of the test object.

FPT calibration
We performed the system calibration in two stages according to (Villa, 2012), the first one is the axial calibration which consisted in take 5 planes in Z axis direction with known increments ∆Z (in this case 10 mm).We project fringes according to (Cogravve, 1999) and (Huntley, 1997) on every plane and calculate a 4 th degree polynomial regression with the phase values in every plane at the known Z positions.
Once we have this relationship, we proceed with axial calibration, to this we used a grid pattern with a 2 cm period and a bright dot in the point that we assume as the origin, see Figure 10.When the calibration is finished, we put the calibration gauges on the reference plane, and project fringes as showed at Figure 11.Table 5 shows standard deviations less than 0.107 mm which demonstrate repeatability of the instrument.The average error increases with the time of use (about one hour) due to projector heating.

COMPARATIVE RESULTS BETWEEN TECHNIQUES
Figure 15 shows the measurement error of different gauges for the implemented technique.SFC has the best accuracy compared with the other three implemented techniques.
As shown in Figure 15, the average error in PST, AST and FP grows according to the size of the gauge because the viewing angle of the camera changes with the height of the measured piece; this suggests a correction of the measuring points with respect to the reference surface.
Figure 16 shows the uncertainties of the implemented techniques.
In this Figure we can see that AST, SFC and FP are comparable, in these measurement systems the uncertainty decreases when the size of the piece is increasing.These systems show low repeatability to reconstruct pieces less than 3 mm in size.SFC also reduces uncertainty by increasing the size of the piece, without breaking its measurement range.
Figure 12.Experimental setup, we used a graduated rail to make the 10 mm increments.

CONCLUSIONS AND DISCUSSIONS
According with (Hocken, 2011) all of the implemented techniques can be improved in the calibration process if consider: image processing algorithms, temperature, intensity of illumination, optical magnification, imaging optics, telecentricity, camera resolution, edge detection technique, material of reference, reference surface tilt, optical magnification of laser sensor, scanning speed and laser intensity.
The software used to fit the points captured by any of the implemented techniques have a greatly influence on the quality of the measured results.We recommend use software with a calibration certificate.SFC, AST and PF, average errors, standard deviations, and uncertainties obtained with the present approach can help the user to identify the tolerances that each technique can achieve and to determine if these techniques can be applied to the control of some quality or measuring The performance of PST shows poor results for the tolerances that were expected based on (Belhaoua, 2010).Because of this, it is bad for quality control purposes, unless improvements in the factors considered on results of PST are done.Tables 6 and 7 describe the advantages and disadvantages of each technique.These   tables let the user to take a decision.For example we can say that PST can be used on computer vision implementations, like robot navigation, but SFC, AST and FP can be used on metrological issues, like reverse engineering.
To calibrate the cameras in PST, AST and FP techniques, we used a printed pattern, this implies an error in the calibration results which affects the error of measurement.The printed patterns introduce an error that can be considered systematic according to (Xiao,210) and therefore corrected, however, the residuals once the correction is applied, must be part of the uncertainty measurement.
Also the calibration process must take into consideration factors such as temperature of the structure holding the cameras, flatness where the printed pattern are pasted, humidity, image processing and others.These and other errors can have a systematic nature and can be corrected, improving the measurement error.These corrections influence the error measurement uncertainty .
The method used by the SFC technique is very different from PST, AST and FP techniques where a pinhole model is considered.The model used was based on (Gonzalez-Barbosa, 2015); however, we must consider that the model could have other errors not considered like the heating of the screen light from Figure 7.
In its favor, this technique uses a certificate pattern instead of a printed one.
Future work will consider the influence of the above errors for smaller measurement error and find major factors of uncertainty affecting the measurements.• Did not has correspondence problems when a dot pattern was projected onto the surface, these projected points simplified the calculation of its 3D position.
• In the absence of a projection pattern; a point in the image of one of the cameras can be associated with several points on the image of the other camera (false match) • If exits a hide point by occlusions in the field of view of one cameras maybe in the other field of view of the second camera this point are visible.
• The selection of correspondence points was made manually can cause mismatches errors.

Active Stereoscopy (AST) Advantages Disadvantages
• Currently used in several commercial measurement systems.
• Presented robustness to measurements made surfaces without texture.
• To establish the relationship between the 2D image points and points in the 3D world, the triangulation method used.
• One of the main limitations of this technique is the problem of occlusion, which occurs when the camera is not able to display the laser beam reflected by the object, since the object geometry.
• The camera reaches detect unwanted reflections generated by the geometric and physical characteristics of the object.
• The reflected laser beam on the object has a thickness and this introduces an impression when detecting the light beam in the image acquired by the camera.
• The laser power affects the sharpness of this on purpose, because if the intensity is not enough, the plane of light will not be detected in the image.371999), Instituto Politécnico Nacional (IPN) particularly at Centro de Investigación en Ciencia Aplicada y Tecnología Avanzada Unidad Querétaro (CICATA-Qro) for the financial support given, and Centro Nacional de Metrología (CENAM) for the facilities Table 7. Reconstruction techniques SCT and FPT with advantages and disadvantages of our implemented systems.

Shape from contour (SCT) Advantages Disadvantages
• Allows rebuild small objects, up to 5 cm.
• When using telecentric lenses, the error is eliminated in perspective vision systems, so that objects can zoom in and out of the chamber without changing the image dimensions.
• System Approximate cost: $ 3,454.99USD • The cost and size of the telecentric lens are inversely proportional to the size of the object being measured.
• When using a lighting system light behind the object to be measured, the camera can detect reflections produced by the geometrical and physical characteristics of the material, making the object appear smaller from the actual size.
• Lighting should be controlled, as excess or deficiency camera is blinded.
• The telecentric lenses are much more expensive than conventional lenses, used in the other techniques.
• If the object surface has concave vertical curves, you will not be reconstructed and the reconstruction system fill in these areas.
• Allows accurate and dense measurements on large objects.
• Robust to noises and sharp edges.
• Gives entire height distribution at every pixel.
• Lighting should be controlled, in excess or deficiency camera is blinded.
• If exist shadows or occlusions the 3D shape is loss.
• Dependent on projector power and resolution.
• For the implementation presented here, requires many images. granted.

Figure 1 .
Figure 1.Gauges used to measure the error of different optical measurement techniques.
and (ISO 15530-3, 2011) Coordinate Measuring Machines (CMM): Technique for determining the uncertainty of measurement Part 3: Use of calibrated work pieces or measurement standards (ISO

Figure 2
Figure2(a) shows the setup with 2 cameras at approximately an angle of 29 o between the optical axis of the cameras and the bar that holds the set up, and a laser that projects a point pattern over the measured gauge.Figure2(b) shows the target point at the central part of the gauge block and the points on the origin plane.Figure2(c) displays the correspondence points projected onto gauges and observed with both cameras.The matching of the points for each image of the left and right cameras generates an arrangement of points that represents the height of the gauge blocks and the height of the pyramid between the first level and the top level.
Figure 4(a)  shows the setup used on this technique.This setup was extracted from the PST setup depicted in Figure2(a) using only the right camera and the laser.
(a) Experimental setup for Active Stereoscopy.(b) Online 3D reconstruction of some meagauges.

Figure 4 .
Figure 4. Active Stereoscopic Technique setup and measurement.

Figure 6 .
Figure 6.Orthographic projection model of a real object into the image plane.
3.4.2FPT experimental setup.The experimental set up consisted of a Marlin F-080 CCD camera, a digital projector SONY VPL-EX5, both 1024x768 pixels resolution, a flat surface and a millimetric rail shown in Figure12.

Figure 11 .
Figure 11.One of the acquired images.3.4.3 Results of FPT Pyramid and calibration gauges reconstruction is showed at Figure 13 and 14.

Figure 15 .
Figure 15.Comparative chart for average errors.X-axis shows the measured objects and Y-axis the average error in mm.

Figure 16 .
Figure 16.Comparative chart for uncertainties.X-axis shows the measured objects and Y-axis the uncertainties at mm.

Table 2 .
Results of Passive Stereoscopy Technique.

Table 3
, the average errors measured with this technique have a positive sign, this was because in the calibration

Table 3 .
Results of active stereoscopy technique.thesameprinted pattern like in PST, but AST only use the right camera, which according to Table2does not have the same left camera intrinsic parameters (focal length, principal point and distortion).The highest standard deviation according to Table 3 is 0.021 mm; this was due to the thickness of the light used.It generates noise that affects the image processing of pieces with dimensions less than 3 mm, see Figure

Table 4 .
Results of Shape from Contour Technique.

Table 6 .
Reconstruction techniques PST and AST with advantages and disadvantages of our implemented systems.