IMAGE-BASED DEFORMATION MONITORING OF STATICALLY AND DYNAMICALLY LOADED BEAMS

Structural health monitoring of civil infrastructure systems is an important procedure in terms of both safety and serviceability. Traditionally, large structures have been monitored using surveying techniques, while fine-scale monitoring of structural components has been done with instrumentation for civil engineering purposes. As a remote sensing technique, photogrammetry does not need any contact with the object being monitored, and this can be a great advantage when it comes to the deformation monitoring of inaccessible structures. The paper shows a low-cost setup of multiple off-the-shelf digital cameras and projectors used for threedimensional photogrammetric reconstruction for the purpose of deformation monitoring of structural elements. This photogrammetric system setup was used in an experiment, where a concrete beam was being deformed by a hydraulic actuator. Both static and dynamic loading conditions were tested. The system did not require any physical targets other than to establish the relative orientation between the involved cameras. The experiment proved that it was possible to detect sub-millimetre level deflections given the used equipment and the geometry of the setup. * Corresponding author.


INTRODUCTION
Deformation monitoring of civil infrastructure systems, or structural health monitoring in general, is an important procedure in terms of both the public safety and the serviceability of the structure.In order to avoid potential structural failure, the maximum loading capacity of the system must be known before its completion, and regularly scheduled maintenance checks must be performed after its completion (Park et al., 2007).Traditionally, large structures have been monitored using surveying techniques (Ebeling et al., 2011;González-Aguilera et al., 2008), while fine-scale monitoring of smaller structural components has been done with instrumentation for civil engineering purposes such as strain gauges (as explained in González-Aguilera et al., 2008;Maas and Hampel, 2006).Both methods have two downsidesdeformation could only be detected at specific point locations, and in the case of failure during the time of monitoring, the area around the object of interest can become hazardous.Thus, this paper will explore the remote sensing technique of photogrammetry for the purpose of fine-scale deformation monitoring of concrete beams.

PREVIOUS RESEARCH
As a remote sensing technique, photogrammetry can provide high-precision non-contact measurements of object(s) or surface(s) of interest with no risk of injury to the operators or damage to the equipment used.Here are some examples from the photogrammetric literature: Mills et al. (2001) used a single small format digital camera attached to a moving crane in order to map a test bed in a pavement rolling facility.Given the used geometry, the experiment resembled near vertical airborne mapping.Despite the undesirable base-toheight ratio, the overall reconstruction root mean square error (RMSE) for the performed experiments was about 2-3 mm; Fraser and Riedel (2000) performed near real-time multi-epoch deformation monitoring of heated steel beams while cooling off in a thermal test facility.Three digital cameras positioned at convergent geometry, and specially designed targets for a such high temperature environment were used to obtain a final precision for the reconstructed object space coordinates of 1 mm; Jáuregui et al. (2003) used double sided targets and measured deflections in steel beams at an RMSE of 0.5-1.3mm in an indoor laboratory.In addition, they also managed to measure the vertical deflections in bridge girders on a highway at an RMSE of 0.5-1.5 mm; Lin et al. (2008) monitored the deformations of membrane roofs with a precision of 1.3-1.6 mm.Their system consisted of two machine vision cameras and one data projector, and it could operate without the use of traditional signalized targets.They achieved targetless relative orientation by defining the scale through imaging the footprint of a reflectorless total station.Also, they managed to generate a point cloud without any physical targets by the means of projecting a pattern onto the surface of interest during the data collection.As stated in their article, the precision could have been significantly improved if more cameras were available.


The next sections of this paper show how a low-cost photogrammetric setup can be used for precise three dimensional (3D) object/surface reconstruction, which could be useful for the deformation monitoring of structural elements in both static and dynamic loading conditions.

PROPOSED METHODOLOGY
The task of deformation monitoring using imagery can be divided into four stages: (1) fulfilment of project prerequisites or system calibration, (2) data acquisition, (3) image processing, and (4) deformation analysis.These four stages are summarized below: 1.The project prerequisites include camera calibration, stability analysis, and estimation of the relative orientation between the involved cameras.In order to assure good quality reconstruction, the cameras are geometrically calibrated (Fraser, 1997;Habib and Morgan, 2003) before they can be used in the project.
It also has to be verified that their internal orientation parameters (IOPs) are stable, i.e. they do not change significantly over time (Habib and Morgan, 2005;Habib et al., 2005).Estimating the relative orientation of the involved cameras requires the collection of signalized target points, and running a bundle block adjustment in order to compute the exterior orientation parameters (EOPs) (i.e. the position X 0 , Y 0 , Z 0 , and the attitude ω, φ, κ) for each camera.
2. The data acquisition stage requires that a setup of multiple off-the-shelf digital cameras and projectors is used.The cameras must be synchronized in order to avoid motion-blur-like errors.The projectors are needed to project a random pattern and thus to provide artificial texture for any objects or surfaces that are homogeneous.This artificial texture is necessary to facilitate the matching of conjugate features between overlapping images as part of the image processing (Reiss and Tommaselli, 2011).
3. The image processing procedure is necessary in order to perform semi-automated object space reconstruction (i.e. the computation of X, Y, Z coordinates for the object(s) or surface(s) of interest).The semi-automated reconstruction includes corner detection in all images, hierarchical image matching between all image pairs, corner tracking of the detected corners common to at least three consecutive images, and a series of multiple light ray intersections using the previously estimated IOPs and EOPs.The reconstruction steps are automated, except for the selection of the region of interest in the images, which is done manually (Detchev et al., 2011a).
4. The 3D reconstruction described above is done for each measurement epoch.The final step in the deformation monitoring scheme is to compute the deflections of certain features of interest relative to a reference datum for all observed epochs (Detchev et al., 2011b;Detchev et al., 2011c).
Note that the photogrammetric system described here does not require any physical targets on the actual object(s)/surface(s) being monitored.The signalized targets mentioned here are only used for the purposes of establishing the relative orientation between the involved cameras.

PHOTOGRAMMETRIC SYSTEM SETUP
A photogrammetric system comprised of multiple cameras and projectors was installed on both sides of a 250 kN hydraulic actuator with an attached spreader beam (see Figure 1) in the structures laboratory at the University of Calgary.The system was to be used for photographing concrete beams (see example in Figure 2) subjected to different loading conditions, where the changing loads would be applied by the hydraulic actuator.A metal frame was designed and built around the actuator (see Figure 3) in order to hoist the cameras and projectors in secure positions above the beam being tested.Observing the top surface of the beam was preferred to observing its longitudinal side, because most of the deformation was naturally anticipated to be along the vertical direction.Four cameras and one projector were attached to each of the two parts of the built metal frame (see the example in Figure 4), thus using a total of eight cameras and two projectors.In order for these attachments to work, the cameras had to be mounted on tripod heads.After the cameras were installed on the metal frame, their EOPs had to be estimated through the use of signalized targets.This is why paper checker-board targets were spread out on the laboratory floor, the concrete beam, and the spreader beam before the start of the experiment (see Figure 5).Since the cameras were rigidly mounted on the metal frame, theoretically one would not expect their EOPs to change for the duration of the experiment.So ideally the bundle block adjustment for the EOP estimation would have been done only once.Nevertheless, due to the long duration of the experiment, and the necessary servicing of the cameras, it was decided that the EOPs should be recomputed before each of the conducted data collection campaigns.Once the testing began however, the targets on the beam had to be removed, and the only targets that could be used for the recomputation were the ones on the laboratory floor (see Figure 6).The targets on the floor were also used for the scale definition in the bundle block adjustment.Several distances between some of the targets were measured with a steel tape, and a distance constraint was implemented in the adjustment solution.It should be noted once again that the projected pattern (see Figure 7) rather than physical targets were used for the purpose of the actual photogrammetric reconstruction.As seen from Figure 7, the projected pattern image added artificial texture to the otherwise white-washed concrete surface, and this made the subsequent matching portion of the data processing possible and also reliable.

EXPERIMENTAL RESULTS
The low-cost photogrammetric system was used for the monitoring of the vertical deflections of a concrete beam subjected to static and dynamic loads.The concrete beam was 3 m long (with a cross section of 30 cm x 15 cm), it was whitewashed, and it had a polymer sheet glued to its underside.Given the hydraulic actuator setup at hand, the spreader beam attached to it was obstructing a large portion of the top surface of the concrete beam.This is why, in addition to observing the visible portions of the beam surface, the cameras also observed 13 (thirteen) 5 cm x 15 cm white-washed aluminium plates attached at 25 cm intervals to the bottom surface of the beam.These metal plates served effectively as offset witnesses to the bottom surface of the actual beam.
The conducted beam deformation experiment was divided into three phases:  Phase Istatic loading based on displacement control: settle the beam on its support by first applying a displacement of approximately 3 mm, and then unloading it in order to return to zero displacement;  Phase IIstatic loading based on load control: apply a load of up to 60 kN;  Phase IIIdynamic loading based on load control: at a rate of 1 Hz apply load cycles from 24 kN to 96 kN for one hour, and then switch to a rate of 3 Hz until failure or a certain number of cycles is reached.
Note that image data was collected during Phase I, Phase II, and the 1 Hz load cycles of Phase III.An example of one of the reconstructed 3D point clouds can be seen in Figure 8.

CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE WORK
This paper dealt with the use of consumer grade cameras and projectors for the deformation monitoring of structural elements.The aim of the conducted experiment was to set up multiple off-the-shelf digital cameras and projectors on a stable metal frame in order to be able to detect deflections in concrete beams during static and dynamic load testing with a hydraulic actuator.After performing semi-automated photogrammetric reconstruction of the visible beam surface and of the full surfaces of all the metal plates, it was shown that sub-millimetre precision for the estimation of the beam deflections could be achieved in object space.Current work involves attempting to approximate the frequency of the beam movement at each reconstructed plate.
In the future, additional cameras will be added to the system in order to monitor the cracks in the concrete.The main task will be to extract the crack borders from each image and track the enlargement of the crack widths in the concrete through image processing methods.

Figure 1 .
Figure 1.Spreader beam (in yellow) attached to a hydraulic actuator (in black and silver)

Figure 2 .
Figure 2. Concrete beam to be used for the experiment (placed underneath the spreader beam)

Figure 4 .
Figure 4. Example of the multiple camera and projector setup on one side of the built metal frame

Figure 5 .Figure 7 .
Figure 5. Example of the distribution of signalized targets on the floor, the concrete beam, and the spreader beam for the relative orientation estimation before the static loading case

Figure 8 .
Figure 8. Example 3D point cloud derived from the photogrammetric reconstruction The reconstructed surfaces in the point clouds for each epoch were first segmented (see Figure 9), and then the Z object coordinates of all the points belonging to the same aluminium plates were averaged.This yielded the centroids of each plate for each observed epoch.Then a reference epoch had to be chosenusually the epoch before any load was applied to the beam, i.e. the zero load epoch, for the static observations, and an epoch at an arbitrary time for the dynamic observations.Next, the Z values of the centroids of each plate for the reference epoch were subtracted from the Z values of the centroids for the corresponding plates for the rest of the epochs.This yielded the beam deflections (δZ) at each plate for each observed epoch, where the deflections for all the plates of the reference epochs were zeros.Example plots for the static and the dynamic data can be seen in Figure 10 and Figure 11, respectively.

Figure 9 .
Figure 9. Example of a segmented 3D point cloud (Note: each segmented plane has a random colour assigned to it)The first three epochs in Figure10represent the zero load state of the beam, where the one marked as 'Rep1' served as the reference, and the other two -'Rep2' and 'Rep3'were used to test the repeatability of the system.The epochs marked as one and two correspond to Phase I, and the epochs marked as three and four correspond to Phase II.It can be seen that the computed deflection values at each plate for the three repeatability sets were within a range of 0.1 mm from each other.

Figure 10 .
Figure 10.Plot of the beam deflections for the static portion of the experiment In Figure 11, the first reconstructed epoch served as the reference, and the following six epochs represent a three second sample interval during the 1 Hz load cycles of the dynamic load testing.The reference epoch was chosen at an arbitrary time, and this is why it does not appear to be on the top, at the bottom or in the middle of the other epochs.At this time of the experiment, a laser transducer recorded that the nominal deflection amplitude for the central plate of the beam was ±4 mm.So the difference of 7.5 mm between the maximum and the minimum deflections at the central plate, observed by the photogrammetric system, closely approximates the true range of motion for the beam at this location.

Figure 11 .
Figure 11.Plot of the beam deflections for a three second sample interval during the 1Hz load cycles of the dynamic portion of the experiment