Relative Accuracy found within iPhone Data Collection

: The use of consumer gadgets such as smartphones can be incorporated in high accuracy LiDAR mapping with a proper assessment of sensors. Apple’s iPhone 13 Pro includes a LiDAR scanner that can create mapping projects with standard point cloud exports using the help of the built-in gyroscope and accelerometer, the iPhone can generate a 3D map by calculating its position and movement as the user navigates around a site. Its ability to create these maps brings up the question of the level of relative accuracy of the iPhone by analyzing and comparing the data to a high accuracy surveying total station and increasing the accuracy by utilizing sensor integration. This paper strives to achieve a better understanding of the relative accuracy found within iPhone data collection.


Introduction
Remote sensing devices have become very accessible through consumer grade gadgets. Oftentimes we find these gadgets utilizing this technology for augmented reality and basic interactions with the world. By adding a virtual layer on top of the real world, consumer grade gadgets like cell phones can utilize a virtual space and connect it to what is in front of the user. Apple's latest iPhone 13 Pro utilizes multiple sensors to present this experience to its users. With the use of gyroscopes, accelerometers, and a built-in LiDAR sensor, the iPhone 13 Pro can sense the world around it and building a virtual layer on the device. The gyroscope is designed to orientate the position of the phone. The accelerometer is designed to measure the speed at which the phone is moving. Huang, E. J., & Onnela, J. P. (2020) explains how the slightest movements from the iPhone can now be detected. The technology is very sensitive to movement, making it appealing for handheld mapping. The use of all these sensors has influenced many third-party developers into creating applications that utilize the iPhones LiDAR sensor to capture mapping data and utilize algorithms with the help of the gyroscope and accelerometer to create a point cloud. These data set can be exported as a point cloud. In addition to the onboard sensors, developers have designed additional sensors that can be integrated with the iPhone to enhance its LiDAR scanning capabilities. The use of sensor integration could allow for higher accuracy positioning while maintaining the convenient workflow of a handheld cell phone capturing high accuracy LiDAR scans. This paper attempts to find the standard relative accuracy of the iPhone's LiDAR system, in addition to incorporating additional sensors to improve set accuracy.

iPhone Components and Sensors
Apple iPhone 13 Pro is a consumer grade device containing several sensors that may be optimized for mapping. While the intended use of these sensors may not be optimized for such a task, a certain threshold can be achieved by understanding the complexity of a few of these sensors. The first sensor of interest is the LiDAR scanner. Beginning with the iPhone 12 Pro in 2020, Apple introduced a LiDAR sensor in addition to three upgraded camera sensors to enhance their video production and image quality. The LiDAR sensor helps with enhancing low lighting conditions and provides the phone with a 3D perspective of the outside world. Integrating augmented reality with the real world allows the iPhone to perceive the world utilizing LiDAR to generate a model within its software. This LiDAR sensor can also be used for mapping thanks to third party developers creating applications that can be accessed by anyone. As described by Luetzenburg, G., Kroon, A., & Bjørk, A. A. (2021), the LiDAR system that was introduced in the iPhone 12 Pro holds a strong potential for mapping smaller objects with 3-6 cm accuracy. These applications allow access to the LiDAR's mapping capabilities by producing a point cloud through a SLAM algorithm that instantly calibrates the LiDAR with the camera sensors visualizing the location and colorizing the points captured by the LiDAR sensors. While moving the iPhone to capture more points and expand the boundary for mapping, the iPhone utilizes its gyroscope and accelerometer. These sensors found within the phone are not optimized for mapping, but rather have a more internal use from within the phone. The accelerometer exists to help users track the speed at which their phone is travelling in, just as they would find their speed while driving a vehicle. The gyroscope allows the iPhone to know its orientation to display certain applications correctly for the user to enjoy. These two sensors help to orient the position and measure the speed at which the iPhone exists at a certain point in time by utilizing these sensors along with capturing real-time data with a LiDAR sensor, any consumer can develop a map. Certain applications are also utilizing the built-in consumer grade GNSS receivers. This is limited to the Global Positioning System, it's ten to fifteen feet, or three to five meters, of accuracy can allow the data to be placed on a particular location on earth. While this may not produce absolute accuracy, researchers can argue that the internal sensors such as the gyroscope and accelerometer can help produce relatively accurate data from the iPhone's LiDAR sensor.

Additional Equipment
The additional equipment used for these experiments include a Sokkia SX-105T Reflectorless Robotic Total Station, a viDoc RTK enabled GNSS Receiver, and a DJI Osmo 3-Axis Gimbal. These devices were utilized and a combination of comparing the iPhone's relative accuracy, enhancing the level of accuracy, as well as providing ground truthing and independent observations not directly linked to the data collection process. As several methods and experiments are underway, the equipment used is vital to ensure a baseline measurement for data being collected and to be compared with the iPhone 13 Pro. Additionally, those sensors that enhance the performance and accuracy of the iPhone require a deep understanding of their intended use for enhancements of the initial stock parts found in the phone. The contribution that this equipment bring allow us to closely monitor the performance of a consumer grade product attempting to achieve higher accuracy mapping.

Proof of Concept
To obtaining high relative accuracy from the iPhone 13 Pro's LiDAR sensor, as well as integrated motion sensors, a small proof of concept is required to achieve an abstract worthy for research. Conceptually, the iPhone could scan any feature with a distinct position and elevation changes, and the measurements taken from one part of the project to another should match the same distance in the real world. As a baseline measurement to benchmark all findings, a high accuracy instrument must be used to establish a record measurement used as the true location of said features. A well trusted and credible piece of equipment would be a robotic total station. Surveyors have utilized this piece of equipment for decades to survey features accurately for engineering designs. As Fenais, A., Ariaratnam, S. T., & Smilovsky, N. (2020) utilized total stations for underground utility work and incorporated it with a workflow that allowed the iPhone to use augmented reality to find information about the utility structures. The initial setup for the project will include two main control points for the total station to establish its local coordinate system, and a common feature of interest that will be measured by both the total station and the iPhone 13 Pro.
The total station will occupy one control point, and the second control point will be at a distance that is further away than the feature of interest to establish a longer baseline for data collection. On the second control point, a reflector prism will be set and observed as the backsight reading for the project. Once observed by the total station, the zenith angle will be set to 0 degrees 0 minutes and 0 seconds to which the total station will be rotated to observe foresight readings on a prism rod of the features in front of it. The feature of interest is a small piece of curb containing back up curb observations and gutter observations in line together. Figure 1 represents the positions of the curb line and gutter line locations the total station observed.

Figure 1. Observation method taken on Curb and Gutter Lines
Once all the observations are taken from the total station, the iPhone is next to collect data. The use of the application "3D Scanner App" allows users to utilize the RGB camera sensor to see where they are pointing their device, and scan using the LiDAR sensor. While scanning data, areas that are measured and recorded are highlighted by a purple outline. Once the entire curb is measured, an LAS point cloud may be exported. This point cloud is imported into AutoCAD Civil 3D along with the observations taken by the surveying total station to begin analysis and comparison. Since both datasets are observed in different coordinate systems, the iPhone's point cloud will be aligned to the total station points. This ensures that when comparing the positions found on the same feature, their coordinates will align since they are now on the same local coordinate system. To perform a three-dimensional alignment, 3 sample points are selected from both the datasets referencing the same location between both datasets, and positioning a point cloud to the total station points as seen in Image 1.

Image 1. The point cloud alignment to points measured with the Total Station
There is no scale adjustment applied to the point cloud as this would disrupt the relative accuracy found in the raw data. The alignment does not disturb the relative distances between all the points as it is a simple 3 parameter transformation to position the points on the same coordinate system as the baseline measurements being assumed as the true recorded coordinates of our future.

Drift Calculation
Upon measuring a large area, the gyroscope and accelerometer begin to experience some degree of error that drifts the data set off course from its correct position. These sensors are considered lower grade, as Guan, Y., & Song, X. (2018) explains, they are a more consumer version of an Inertial Measurement Unit (IMU).
Considering that most real-world applications include large scan trajectories, it is important to recognize the amount of drift associated with the distance traveled when utilizing the iPhone's LiDAR sensor. An experiment to determine the position of certain points on the output point cloud in relation to the true positions in real life can determine how much error is accumulated due to drift over a set distance.
A project site with a fixed 200-foot length will determine the accuracies of the iPhone with all its sensors in relation to a baseline data set from the surveying total station. The initial measurement begins with the use of a measuring tape. This tape is extended to approximately 200 feet (60.96 meters) to give a visual understanding of the length of the projects. The 200 feet will be represented in the engineering stationing terms 2+00, where every station is equivalent to 100 feet (30.48 meters). At every 10 feet, 0+10 station, which is equivalent to 3.05 meters, a wooden stake shall be placed as a marker, extending approximately 1 foot above the ground. This is to help ensure that the LiDAR sensor observes these points, and they are not lost within the terrain. After the 21 stakes are placed along the 2+00 project, the measurements utilizing the survey total station begin. The total station occupies station 2+00. A backsight prism is set Gutter Line Curb Line

= Observation
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France up on station 0+00 with an assumed coordinate of 5000, 5000, 100, where the first coordinate is Northing (Y), second is Easting, (X), and third is Elevation (Z). The total station observes a measurement to the backsight, and the horizontal distance is 200.05feet and the vertical distance is 2.04 feet. Given the measured distance, we can now assign coordinates to station 2+00 at 4800.05, 5000, 97.96. Using a foresight prism, all remaining stakes are measured to ensure an accurate baseline observation utilizing the surveying total station. once all 21 points have recorded coordinates, the iPhone 13 pro can begin scanning the 200-foot site. starting on station 0+00, the user begins scanning using the LiDAR sensor, and the phone's gyroscope and accelerometer measure the motion of the phone as it moves towards station 2+00. When arriving at the end of the site, the user ends the data collection, and processes the point cloud in Image 2.
Image 2. The point cloud from the iPhone's LiDAR Sensor The data sets are brought into CAD software where they are analysed and compared to each other. An alignment of the point cloud is necessary to maintain the same local coordinates of the surveying total station, Figure 2 shows what the alignment looks like at stations 1+80, 1+90, and 2+00. Aligning stations 0+00, 1+00, and 2+00, without scaling the point cloud, ensures a relative fit to the total station data.

Emulating UAS Workflows
Oftentimes developing proper methods for enhancing accuracy in and research requires emulating workflows found in similar devices. Unmanned Aircraft Systems (UAS) utilize several types of sensors to enhance their ability to capture data in a more reliable and accurate manner. Modern day UAS workflows include the use of ground control points, sensor stabilization to minimize shakiness and instability, and enhancing Global Navigation Satellite Systems (GNSS) capabilities to achieve centimeter level accuracy with respect to absolute positioning.
The use of ground control points allows users to utilize independent observations, typically with the use of GNSS receiver, to observe satellites and process corrections for their position on a marked control point that can be visible within the scan. This position can be georeferenced in the data set to position the point cloud in a more accurate location. This transformation typically requires a relatively accurate data set, however there can be some minor adjustments that rescales the project.
Additionally, earlier generations of UAS mapping included a fixed mounted camera sensor that mimicked the exact movements of the UAS while it was flying. This compromises the camera's ability to capture high quality imagery as the UAS's flight is disturbed by the camera sensor's ability to focus on its subject matter, in this case the terrain below it. To combat this oversight, the workflow now includes the use of a three-axis gimbal. This gimbal allows for any type of payload to be stabilized and maintain its exact position, regardless of the movements of the UAS. The three-axis gimbal's job was to maintain and compensate any movements from the UAS and provide the payload with this compensated positional adjustment to maintain a fixed location, or minimize any unnecessary movements caused by the flight. Altan, A., & Hacıoğlu, R. (2020) found that this allowed for much cleaner and higher accuracy imagery as the camera sensor could focus on the subject matter below it.
The final advantage found in modern day UAS is built-in capabilities to connect to a Real-Time Kinematic (RTK) enabled corrections while flying missions. Its controlled segment is added to the solution in real time while the UAS navigates through its mission. This attaches a very high accurate geotag to imagery and it minimizes the need for additional independent observed control points. Ekaso, D., Nex, F., & Kerle, N. (2020) found this also helped to maintain calibration between camera positions as the level of certainty for the geotags increased significantly allowing for calibration to be more confident. A direct comparison of ground control points that weren't added to the solution, check points, found root means square errors at about 0.1 feet (3 cenimeters) in ideal conditions.

Independent Control
Just as UAS workflows incorporate the use of ground control points, the same concept is implemented with the iPhone's LiDAR scanning system. As discussed in Long, N. Q., Goyal, R., Bui, L. K., Cuong, C. X., Canh, L. V., Minh, N. Q., & Bui, X. N. (2021), the key indication for the control point is the target laid out as this is the feature found in the UAS's imagery. High contrasting colors, such as white and black, make it easy for users to identify the location of the center of the target, being the position of the control point. Utilizing software, users can georeference the location of the pixel in which the coordinates from their independent observations can be assigned. In this case of LiDAR, as seen in ground LiDAR units and airborne LiDAR units, high contrasting paint or stripping must be present for the LiDAR intensity to pick up the edge of the control point. The edge of two high contrasting tape markers will indicate the center position at which the georeferencing position should be identified. These control points will assist in stitching several scan projects from different locations on the job site. The control points also position the project in a coordinate system and provide higher absolute accuracy with respect to positioning on The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France earth. While there is still a heavy reliance on relatively accurate data, these control points can still assist in maintaining some degree of accuracy, or at the very least contribute as one of many factors that can increase the accuracy obtained from an iPhone's LiDAR system.

Sensor Stabilization
The stabilization of any sensor is crucial in data collection. Often sensor compensators may not pick up on irregular movements and register false information that can cause drift or misalignment datasets. Just as UAS use a three-axis gimbal to stabilize their sensor, similar products are designed for smartphones to stabilize them. These smartphone gimbals are typically designed for photography and smooth transitional videography. However, the use of this gimbal can also stabilize the position of the phone when capturing data as the phone remains in a locked position and only moves with the movement of the user rather than the irregular movements of the user's hands. While a user has an irregular walk possibly going up or down terrain, or while shaking the phone in ways they themselves may not notice, the phones gyroscope recognizes and interprets this in an accurate fashion and skews data sets which can cause misalignments. The use of a gimbal can eliminate some of this error and produce higher quality data as the gyroscope and accelerometer require less attention, as the movement of the phone is more stable and positioned in the same location regardless of irregular movements by the user.

Enhanced GNSS Capabilities
Third party developers have managed to utilize the LiDAR sensor by creating applications that allow users to map out their surroundings with the LiDAR sensor. These developers have attempted to utilize the built in GPS antenna to give a general location to the scan. While some developers proved to be successful in tagging locations for their projects, it is impossible to increase accuracy for the project utilizing the built in GNSS given its inability to perform real time corrections. However, with the incorporation of an RTK enabled GNSS antenna in Image 3, users can connect via Bluetooth to this antenna and attach the antenna and receiver block to the back of their iPhone phone case.

Image 3. viDoc RTK GNSS for iPhone
As the phone moves to collect data, the RTK enabled antenna moves as well and records high accuracy positioning with real time corrections. This emulates UAS with RTK enabled antennas while flying and capturing high accuracy geotags. The science of enhanced GNSS capabilities is replicated onto the iPhone's LiDAR system, to provide higher accuracy LiDAR scanning with the help of high accuracy GNSS positioning.

RESULTS AND CONCLUSIONS
Upon extensive testing of Apple's iPhone 13 Pro's ability to perform LiDAR scanning, verdicts have been achieved in the following areas:

Resulting error in Drifting
After aligning the point cloud to the surveying total station's points, the user can extract the coordinates at the top of all 21 stakes, measured by the iPhone 13 Pro's LiDAR sensor. After station 0+10, errors reach 0.10 feet (3.0 centimeters) in the horizontal and 0.05 feet (1.5 centimeters) in the vertical. At station 0+50, errors reach 0.2 feet (6.096 centimeters) in the horizontal and 0.15 feet (4.6 centimeters) in the vertical. At station 1+00, errors reach 0.5 feet (15.2 centimeters) in horizontal, and 0.2 feet (6.1 centimeters) in the vertical. At station 2+00, the root means square error reaches 1 foot (30.48 centimeters). Therefore, when utilizing just the built-in sensors in the iPhone 13 Pro's, the standard scanning capabilities, users can expect 1 foot of error for every 200 feet of scanning, which can be seen in Table 1. This is a clear demonstration of how the onboard sensors could use some assistance with some additional sensor integration. While it is an impressive achievement for a consumer grade cell phone to have this level of accuracy, to perform survey grade mapping, additional sensors will be required. Sta.

Improvement with Sensor Integration
The onboard sensors are not optimized for mapping with the iPhone. Therefore, introducing exterior sensor integration similar to that of the UAS Workflows could help enhance accuracy of datasets within the iPhone's LiDAR system.

Independent Observations:
Before starting the scan for the iPhone, this time station 0+00, 1+00, and 2+00, were utilized as ground control points and used to create a best fit for the point cloud. This adjusted the point cloud and scale this to fit these points best as possible. While this does manipulate the original point cloud being produced, the concept of independent observations is designed to help closely replicate the true location, and these adjustments and modifications to the point cloud are necessary to fix any potential errors. In doing so, after station 0+10, errors reach 0.07 feet (2.1 centimeters) in the horizontal and 0.05 feet (1.5 centimeters) in the vertical. At station 0+50, errors reach 0.12 feet (3.7 centimeters) in the horizontal and 0.10 feet (3 centimeters) in the vertical. Station 1+50 had the largest root means square error at 0.75 feet. This is because the relative positioning of the points still contained a lot of drift, and at areas that were the furthest from the initial start and in between control points, would experience the most amount of error in comparison to the rest of the project. This shows a lack of attention for the entire site as this method of data collection needs improvement. The results of this experiment have been plotted and can be seen in Figure 3.

Three Axis Gimbal:
Stabilizing the iPhone was a major advantage as there would be less compensation necessary from the accelerometer and gyroscope. Less compensation means a more accurate position while collecting data from the LiDAR sensor, which should yield higher accurate point cloud. Reverting from the last workflow, no ground control points were used in this trial, only the threeaxis gimbal to stabilize the phone while observing the 21 stakes. After station 0+10, errors reach 0.05 feet (1.5 centimeters) in the horizontal and 0.02 feet (0.6 centimeters) in the vertical. At station 0+50, errors reach 0.1 feet (3 centimeters) in the horizontal and 0.1 feet in the vertical. At station 1+00, errors reach 0.3 feet (9.1 centimeters) in horizontal, and 0.15 feet (4.5 centimeters) in the vertical. At station 2+00, the root means square error reaches 0.5 feet (15.2 centimeters). Therefore, when utilizing a three-axis gimbal with the built-in sensors in the iPhone 13 Pro's, users can expect 1 foot of error for every 400 feet of scanning. By simply increasing the stability of the phone, we can see a drastic decrease in the drift error of the iPhone.
Much of the drift can be contributed to a non-steady hand and non-steady movements by the user. Because the surveying total station is in a fixed location, it is easy to see how its measurements are superior to any handheld device. However, the ability to stabilize a handheld device utilizing just a simple threeaxis gimbal solidifies the UAS workflow of utilizing a gimbal to stabilize imagery for higher accuracy aerial mapping. The same should be applied for iPhone LiDAR mapping.

RTK enabled GNSS:
The final sensor integration method is enhancing the GNSS receiver found in the iPhone. A consumer grade GPS is useful for navigating in a vehicle, however for mapping purposes, the use of a higher RTK-enabled GNSS receiver is necessary. Using Pix4D's viDoc allowed the use of RTK correction for our positioning. This measurement was based on the GNSS position and the IMU built into the case. The application uses the RGB camera sensors to follow and reconstruct the project while in their application. This allows for tracking of the site and helps maintain the relative accuracy of the project. Like the last experiment, all other integrations have been removed and the only adjustment from the initial workflow is the use of the RTK antenna. After station 0+10, errors reach 0.02 feet (0.6 centimeters) in the horizontal and 0.02 feet (0.6 centimeters) in the vertical. At station 0+50, errors reach 0.1 feet (3 centimeters) in the horizontal and 0.1 feet (3 centimeters) in the vertical. At station 1+00, errors reach 0.18 feet (5.5 centimeters) in horizontal, and 0.2 feet (6.1 centimeters) in the vertical. At station 2+00, the root means square error reaches 0.3 feet. Therefore, when utilizing an RTK enabled GNSS antenna and the built-in sensors in the iPhone 13 Pro's, the standard scanning capabilities, users can expect 1 foot of error for every 666 feet of scanning.
Out of all these sensor integrations discussed, incorporating an RTK enabled GNSS receiver is the most effective with respect to capturing accurate data. While 0.3 feet (9.1 centimeters) is not a perfect answer, it does increase the level of accuracy by 70% from the initial sensors found in the iPhone's system.

Visual Depictions
All the data sets utilized the same camera sensor and LiDAR sensor when performing the scan. This means the at first glance they all have the same resolution in respect to the colorization of the point cloud and the same density of points captured by the LiDAR sensor. This point clouds are all developed using the same SLAM algorithms developed by the creator of the application used for collecting data.

Conclusion and Discussions
Given that the initial purpose for the iPhone 13 Pro's sensors is for augmented reality, it is assumed that these sensors are not optimized for mapping. However, after careful analysis and comparing the data to high accuracy total stations, certain small applications could be appropriate with the use of the iPhone 13 Pro. Sensor integration allows for higher accuracy positioning with the assistance of some of the downfalls of the gyroscope, accelerometer, and built-in consumer grade GPS. These additional sensors can improve positional accuracy as well as relative accuracy of the datasets that come out of the iPhone, without compromising its unique mobility and ease of use. Apple's flagship phones still need more research development to achieve a survey grade measuring instrument, however, given the existing hardware inside of it, this seems surely attainable soon. It's important to understand the deep impact these additional sensors have on the data collection process. Some may only affect change to the accuracy by a small margin, however given the cost of some of these sensors, one can imply that minor and affordable changes to the data collection process could significantly improve the results. In the case of using the gimbal, the user isn't required to speed as much on The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B2-2022 XXIV ISPRS Congress (2022 edition), 6-11 June 2022, Nice, France additional sensor integration, or even the same about of time calibrating everything as it is just a simple alteration to the existing work being done. With the case of the GNSS upgrade, the user must set up and calibrate everything on the application provided by the developer. The biggest drawback to this in researching is the ability to control all aspects of the project. For example, having a black box system does provide consistent parameters for data collection, however, limits the amount of adjustment that could be made for each scan. This limitation take analysing algorithms out of the discussion as users are unable to see what methods the developers used to capture this data.