REFINEMENTS AND INTRODUCTION OF PHOTOGRAMMETRIC ELEMENTS TO AN EXPERIMENTAL TRILATERATION BASED OPTICAL METROLOGY SYSTEM

OPTIMUM is a novel optical coordinate measurement system designed to determine the location of omnidirectional targets within a large volume, the previous version of the system could determine the targets location with an uncertainty of 50 × 10−6 m. This paper describes some of the limitations of the original embodiment and changes being developed to address them. Version 2 of the system aims to solve the limitations of version 1 by integrating photogrammetric processes into the design and control processes of the system, along with redesigning the mechanical and optical relationships. The redesigned instrument will result in the ability to fully automate the initial calibration process and to allow for real-time processing of the measurement algorithms.


INTRODUCTION
The National Physical Laboratory (NPL) is currently developing a novel optical coordinate measurement system (OP-TIMUM) designed to have: a measurement volume of 10 m x 10 m x 5 m; an accuracy at least as good as a laser tracker; inherent self calibration ability at the point of use; direct traceability to the International System of Units, the SI; and the ability to routinely provide rigorous measurement uncertainty evaluation for all points measured (Hughes, 2017) This paper describes the limitations faced by version 1 of the system and the solution presented to overcome them for version 2 of the system.
The system is shown conceptually in Figure 1. It comprises of a number of sensors that surround the measurement volume and retro reflecting targets in the form of glass spheres, located at points of interest. The glass spheres were made of glass with a refractive index close to 2.0 which has the property of acting as an omnidirectional retroreflector. The sensors were designed to use infrared frequency scanning interferometry (FSI) to simultaneously measure the distance between the multiple sensors and multiple targets. The distance data is then analysed using multilateration -fitting the parameters of a mathematical model of the measurement setup to the observed distance data to calculate coordinates of the targets and sensors in a unified coordinate frame, the frame being defined using the positions of three of the targets. The uncertainty in the distance measurement, estimated at the time of measurement, is propagated through the fitting process to provide a rigorous estimate of model parameter uncertainties in accordance with the ISO guide to the expression of uncertainty in measurement (GUM). Figure 1 shows the configuration of the multilateration system observing range measurement to omnidirectional targets.
Version 1 of the OPTIMUM system was constructed of 4 sensors, each sensor comprised of a spatial light modulator (SLM) and associated optics to generate multiple FSI laser beams, each directed towards an individual target, a camera to view the scene in red laser illumination and software to locate * Corresponding author Figure 1. OPTIMUM instrument setup to measure a large measurement volume containing multiple targets and sensors simultaneously the targets in the image. The image coordinates of the targets were used to determine the direction in which to send FSI laser beams to make the range measurements. The laser illumination was used to increase the visibility of the targets in the camera image. A single lens was used both as a projection lens for the infrared FSI laser and the visible red laser while also acting as the camera's imaging optics. The prototype system was tested against an established metrological system in the form of a laser tracker, the results showed uncertainties that were comparable to the same order of magnitude to the laser tracker (50 × 10 −6 m) (Campbell, 2016).
FSI differs from the traditional interferometer, it uses a tuneable laser source to sweep the laser frequency over a range, in this case, of several THz. The interference signal produced is analysed in the frequency domain to determine absolute distance to the target reflector. The frequency domain analysis enables multiple measurements to multiple targets to be made simultaneously with the same sensor.
The SLM (Hamamatsu X13139) is a reflective liquid crystal on silicon device with 1280 x 1024 pixels that can be individually addressed to adjust the phase retardation with 8-bit dynamic range. The device and its associated optics, shown in figure 2 are configured to allow modulation of the phase of an incident plane wavefront such that an arbitrary intensity pattern can be produced at the focal plane of the Fourier lens. The wide-angle projection lens then projects the intensity pattern into the measurement volume. In this way individual beams can be directed to each target in the volume by generating a pattern of spots in the image plane of the Fourier lens. The Gerchberg Saxton algorithm [ (Gerchberg, 1971) and Figure 3] is used to calculate the phase modulation required for the required intensity pattern. The Gerchberg-Saxton algorithm results in an intensity pattern with a resolution equal to the number of pixels on the device. This implies a limited angular resolution for the projected FSI laser beams. The SLM optics has an approximate field if view of 70 • resulting in a physical angular resolution of 0.055 • pixel −1 x 0.068 • pixel −1 . The Gerchberg Saxton algorithm has been implemented on a Nvidia Jetson AGX Xavier embedded GPU system, achieving an updated frame in 20 ms allowing for close to real time beam steering.
The angular resolution of the SLM requires each beam to be diverging slightly in order to eliminate blind spots in the measurement volume. If we assume a total IR output power no higher than 10 mW to ensure eye safety, we can estimate the maximum range at which a sufficiently powerful return signal is obtained from a target for a good quality measurement to be made. However, this calculation is complex and depends on a number of factors including the number of targets being measured simultaneously, the refractive index of the target, the wavelength range over which the FSI beam is swept, the tuning rate of the laser and other factors. The FSI signal detection is a heterodyne process that, in practice, requires only a few picowatts of return power. Taking all factors into account, and based on practical experience, we estimate that the maximum range that can be reliably measured is > 15 m for up to 10 targets.
The coordinates of each target, T k , and sensor, Sj, can be estimated by solving equation 1 if N ≥ 4, M ≥ 6 and six coordinates associated with three different targets or sensor heads are constrained to define the coordinate frame.
In equation 1 the j th sensor is at position Sj, the k th target is at position T k , the distance between the sensor and the target is d jk and wj is the length offset associated with each sensor's optical path length. The model can also have other systematic errors added as appropriate for further improvement.
The combination of photogrammetry and multilateration allows the system to produce a refined set of results compared to the version 1 system. We expect this set of refinements to result in a system that can robustly and reliably determine multiple targets to (50 × 10 −6 m) over a 500 m 3 volume at data rates of several frames a second.

CURRENT LIMITATIONS
Although the initial principle testing of the OPTIMUM system proved successful, it involved a lot of manual measuring and post-processing of the data. This section outlines the limitations. Version 2 of the OPTIMUM system is now in devel-opment, aiming to improve on the success of the initial proofof-principle testing. The original system suffered a number of limitations and drawer backs:

Illumination of the Targets
The through-the-lens laser illumination used to highlight the transparent targets in the camera image was chosen on the basis that the reflection from the refractive index 2 ball lens targets is limited in angular range to a narrow cone centred on the line joining the source of the illumination to the centre of the target. Whilst efficient in terms of return signal collected by the lens from the targets, this arrangement resulted in a blind spot and noise in the camera image due to back-scattered light as shown in Figure 4.

Dual Purpose Optics
Using the same lens for both projection and imaging conveniently resulted in both sub-systems sharing the same optical axis and hence natural alignment. However, in practice, the imaging optics required a large depth of focus and hence high F number whereas the projection system required a low F number. It was not possible to achieve both with a single lens.

Mapping Camera Pixel to SLM Pixel
Although the imaging and projection sub-systems shared the same optical axis and hence two rotational coordinates, it was still necessary to know the translation of the SLM coordinates relative to the camera's pixels as well as any distortion of one coordinate system onto the other due to additional optics, size differences and tilts etc. Determining the relative orientation of the SLM and camera required a manual procedure involving tracing each IR beam to the target using an IR card whilst operating in the dark.

Optical Path Length Distortion
The optical path length between the sensor and the target includes the projection optics. Since no optical system can realistically be free from aberrations, the measured distance to each target was subject to an error related to the direction of the beam through the optics. This was not compensated in the version 1 system as no method for orienting the sensor was implemented. Measurement errors caused by the projecting lens are considered a large contribution to the uncertainty achieved in the initial measurements.

Range to Target Correspondence
Establishing the correspondence between each range measured by a sensor and its associated target was implemented using a manual trace along the IR FSI beam using an IR card, and manually adjusting the beam direction using the SLM until it hit each target. Like the SLM to camera mapping procedure, this had to be carried out in the dark to ensure visibility of the IR beam on the card.

SOLUTIONS
This section describes the changes to the system design proposed to correct the limitations listed above. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2020, 2020 XXIV ISPRS Congress (2020 edition) Figure 4. Subsection of a camera image highlighting the noise and blind spots present when viewing the glass sphere captured through the combined SLM camera lens with illumination being provided by laser light projected through the imaging lens. Figure 5. Subsection of a camera image showing the glass sphere captured through a conventional camera lens with illumination provided by a ring light, unlike image 4 the glass sphere can clearly be seen with little backscatter or noise present. Spherical retro reflective photogrammetry targets can also be seen in this image.

Illumination of the Targets
The back scatter into the camera caused by the through-thelens configuration has been eliminated using a more conventional LED ring light approach. The image of Figure 5 clearly shows the image of a target obtained with a ring light operating at 650 nm. Further work is ongoing with a particular focus on the design of the light and choice of colour to optimise the efficiency of the return signal.

Dual Purpose Optics
Replacing the single, dual-purpose lens with a dedicated lens for FSI beam projection and imaging allows us to optimise the properties for each function. However, this does introduce the requirement to know the relative six degree-of-freedom orientation of SLM projection and camera coordinate systems, as well as the distortion parameters of both lenses, so that the distortion corrected camera coordinates of a target centroid can be mapped to a direction vector, in SLM pixel coordinates, for the SLM to produce a FSI beam directed accurately to a target.

Optical Path Length Distortion
The version 1 of the system achieved uncertainties of 50 × 10 −6 m despite using equation 1 without the wj term present. The main contributing factor to wj is the FSI range distortion through the SLM projection optics, in order to determine the range distortion the optical axis vector needs to be calculated, which equates to finding an accurate exterior orientation of the SLM. Initially the camera and SLM will be mechanically constrained via an environmentally stable material, ensuring the relationship between the nodes of the 2 lens stay aligned during measurement. The initial geometry will then be precisely measured using the camera as a point of origin, with the corresponding orientations being used for all further calculations.

Mapping Camera Pixel to SLM Pixel
Mapping the camera and SLM pixels can be done using current photogrammetric epipolar geometry techniques. Due to the uncertainty in the relative orientation geometry, epipolar projections from the camera to the SLM will be a band. To reduce the width of the band the uncertainty in the relative orientation needs to be reduced, this can be achieved as the SLM image search locates more targets. The location of 5+ targets allows for the relative orientations of the sensors to be calculated and corrected if the new positions provide better uncertainty (Luhmann, 2014). The location of the target coordinates in the SLM's image can then be determined using an epipolar search algorithm (figure 6) using the following steps: 1. Locate desired target in camera image at position x , y 2. Project the epipolar line onto the SLM image and convert to band of a width corresponding to the geometry uncertainty 3. Use FSI to scan a line the width of the band along the length of the band until a target is located 4. Use FSI to scan a line the width of the band up a line orthogonal to the band where the target was located to locate the target again 5. Target has now been located in the SLM image at x , y The epipolar geometries used to project from camera and SLM are standard photogrammetric practices, however unlike standard photogrammetry algorithms which assume the locations of all targets in all camera images are known before applying the correspondence algorithms (Maas, 1994) these algorithms have to locate and evaluate the locations of the target in the SLM image plane as the correspondence algorithm runs.

Range to Target Correspondence
The separation of the camera and SLM also presents a solution to the target correspondence problem, a camera on each sensor head will mean a minimum of 4 camera will be imaging the measurement volume. Introducing photogrammetry will allow for the coordinates of the targets to be measured in the world frame (XYZ). For every acquisition, the FSI outputs an array of ranges, these can then be matched to an array of ranges calculated based on the target positions from the photogrammetry, allowing for the automation of the multilateration process. Augmenting the additional data provided by the imaging system into the multilateration process may also become beneficial to the robustness of the solution.

CONCLUSION
The photogrammetric techniques incorporated into version 2 of OPTIMUM will allow for the instrument to drop it's dependencies on manual setup operations and long post processing workflows. The separation of the optics and introduction on a new illumination system reduces the backscatter and noise that was previously affecting the system. The introduction and innovation of photogrammetric techniques will allow for the full automation of mapping from camera pixel to SLM pixel whilst also allowing for real time determination of the orientations. Uncertainties can be reduced by including the wj term in equation 1. The most significant systematic error that is not currently compensated is the optical path length error introduced by the projection lens. We expect to be able to compensate this provided we can determine the orientation of the SLM. Then we can apply a correction based on the direction of the beam through the lens which will allow for a reduction in uncertainties from the previous system. Using known photogrammetric techniques also solves the correspondence problem from the version 1 system, using multiple cameras to calculate the 3D positions of targets and their associated ranges to the projection lens.
The combination of the proposed solution will allow for the next iteration of the system to fully automate its calibration procedures and for it to acquire measurement in real time. This paper would have liked to have presented verification of some of the techniques introduced into this paper, unfortunately covid-19 has made this impossible. Future work in this topic will see laboratory verification and refinement of the proposed solutions.