PANORAMIC IMAGES, 2D FEATURE-BASED AND CHANGE DETECTION METHODS FOR THE DOCUMENTATION OF CONTAMINATED CRIME SCENES

: This paper aims to propose and validate a methodology which can support forensic technicians while documenting a crime scene, after a contamination event (either accidental or deliberate) has changed its original appearance. Indeed, investigators need fast and automated tools to detect changes that occurred at a scene over time, and solutions to this problem are still an open issue. The contribution of the paper for addressing this need is twofold. First, a new methodology is introduced, that exploits panoramic images acquired with the Ricoh Theta SC camera, and 2D feature-based methods. The core idea is that SIFT features inherently contain scene information and, thanks to their good stability and invariance, can be exploited to detect possible changes that occurred at a scene surveyed with multi-temporal images. Second, in order to evaluate the performance of the proposed methodology, a reference approach is applied, based on state-of-the-art change detection algorithms (MAF/MAD), originally developed for remote sensing applications. Both methods are tested by simulating a typical crime scene, and a contamination event at the Crime Scene House (UK).


INTRODUCTION 1.1 Context
Crime scene documentation is a fundamental task which has to be undertaken in a fast, accurate and reliable way.Indeed, it is paramount to highlight any evidence, that can be used to ensure justice for victims and guarantee the successful prosecution of perpetrators.Traditionally, police officers, forensic technicians and researchers have focused on documenting a crime scene and removing and securing evidence.However, during this process, valuable evidence may be lost, moved from its original position or accidentally destroyed.The contamination of the original appearance of crime scenes, whether it is accidental or deliberate, is still a pressing issue: although crime scene personnel are becoming even more sensitive in terms of contamination prevention, the high number of personnel at the scene and the pressure of the time limitations imposed in some forensic investigations, may result in evidence displacement or removal.Proof may also be intentionally taken away from the scene, to erase any traces left behind e.g. by a perpetrator.Furthermore, due to time pressure, scene examiners may fail to adequately identify and document all of the elements that are present (Dutelle, 2016).Indeed, the position and location of evidence is crucial to an investigation and it is essential that such information is accurately recorded and documented.For instance, in cold case reviews, where the police revisit crime scenes years after the events, imagery captured in the first hours after the crime has been discovered, can be crucial in reconstructing the original appearance of crime scenes and highlighting any contamination that occurred.

Literature review on crime scene documentation
In this context, the most recent approaches of documenting and presenting crime scenes can today exploit a wide variety of active and passive optical sensors and non-contact 3D imaging methods in order to address these challenges.Many change detection techniques are also available, to detect the differences occurred over time, given a set of multi-temporal, possibly coregistered, images of the same scene.

Active and passive 3D imaging systems
Active sensors, such as Terrestrial Laser Scanners (TLS), have proven to be successful in a wide range of applications, such as heritage documentation, reverse engineering, land mapping, and, lately, forensic science.Since traditional forensic documentation methods can be costly and time consuming without ensuring a detailed scene digitization, in the last decade 3D technology has provided a valid and valuable resource.Indeed, the use of laser scanner technology for the documentation of crime and collision scenes has significantly grown, as shown by the extensive body of literature in this field (Buck et al., 2013;Liscio et al., 2016;Dustin and Liscio, 2016).Among the image-based approaches, a comparative study exploiting a 360-camera is proposed in Sheppard et al. (2017).The authors investigate the accuracy of the adopted system in a mock crime scene and assessed the results in comparison with forensic documentation standards.The image-based method proposed by Zancajo-Blanquez et al. (2015) is instead applied to the forensic infography technique, which aims to relate all the evidences in order to determine and demonstrate the facts related to a crime scenario.Another novel application can be found in the work of Thali. (2000), which successfully applied photogrammetric techniques for the documentation of forensicrelevant injuries.

Change detection methods
Detecting changes in images of the same scene acquired at different epochs has seen an important development in a wide range of disciplines such as video surveillance, remote sensing, medical diagnosis, civil engineering, cultural heritage, disaster management and driver assistance systems.Currently, the application of change detection methods is very limited in forensic science, focusing mostly on forgery activities and digital crimes.In Sharma and Dhavale (2016), a review of passive forensic techniques for the detection of copy-move attacks on digital videos is illustrated due to the urgent requirement of authenticating the integrity of the contents of digital videos for use in court.Bravo-Solorio and Nandi (2009) have proposed a passive forensic method for detecting duplicated regions affected by reflection, rotation and scaling.To achieve this, an overlapping block of pixels are re-sampled into log-polar coordinates, and then summed along the angle axis, to obtain a one-dimensional descriptor invariant to reflection and rotation.A brief description about the use of change detection methods in human identification through facial recognition and imagery analysis is presented in Oxlee (2007).The author describes the identification of a woman by combining different kinds of algorithms.

Paper objectives and contributions
This paper aims to propose and validate a methodology which can support forensic technicians while documenting and recording a crime scene and any possible contamination that could have changed its original appearance.Previous investigations performed by the authors (Abate et al., 2017) showed promising results, achieved by using a low-cost panoramic camera and photogrammetric techniques for 3D scene reconstruction.However, the original approach had some limitations, mainly due to (i) the noise affecting the generated 3D point cloud, leading to the detection of some false evidence; (ii) the significant time required for the entire photogrammetric pipeline, including image pre-processing, 3D reconstruction and final contamination recording.In order to avoid these issues, a different and improved approach is presented in this follow-up research that exploits panoramic images and 2D feature-based methods.In particular, it provides investigators with an effective and rapid means to:  document changes that occurred between the original and the contaminated crime scene;  retrieve the original (i.e.pre-contamination) positions of objects and evidence;  visualize them in a simple and intuitive way.In order to evaluate the performance of the proposed methodology, a more traditional one is applied to the same dataset.This reference approach is based on the adoption of state-of-the-art change detection algorithms, originally developed for remote sensing applications, and applied here for the first time to the documentation of contaminated crime scenes.

Developed methodology 2.1.1 2D feature detectors, descriptors and matching
A variety of algorithms have been developed by the computer vision community to automatically extract a large number of points or regions of interest from images.Indeed, 2D features detection and matching are an essential component of several photogrammetric and computer vision applications (e.g.image classification, object recognition and matching, 3D scene reconstruction, motion tracking, etc.) and many efforts have been recently focused on evaluating their performance (Wu et al., 2013;Apollonio et al., 2014;Işık and Özkan, 2015;Karami et al., 2015).Once a number of salient features are detected, they can be mutually compared, in order to establish feature correspondences across different images.Since the element to be matched is an image feature, these algorithms are usually referred to as 2D feature-based methods.Normally, they require a two-step procedure: first, interesting features are detected (by feature detectors) and associated with feature descriptors for their representation (by feature descriptors); second, the corresponding features are determined using similarity measures involving the feature descriptors (by feature matching).With regards to the first step, scale-and affine-invariant region detectors are usually preferred, in order to deal with the significant geometric and photometric variations between the images under wide baseline configurations.Among these solutions, nowadays the most popular operator is the Scale Invariant Feature Transform (SIFT) algorithm (Lowe, 2004).It provides highly distinctive features that are invariant to image translation, scaling and rotation and partially invariant to illumination changes and affine or 3D projections.The operator follows a four-step strategy, comprising scale-space extreme detection, keypoint localization, orientation assignment and keypoint descriptor creation.Thanks to its good stability and invariance, SIFT method (and its variants) is the most adopted solution to the problem of matching images that evolved by some transformations and distortions.

Feature-based method for contamination detection
The methodology developed for the rapid documentation of contaminated crime scenes is summarized in Figure 1.The pipeline takes as input (i) a set of panoramic images of the crime scene, captured when the crime is first discovered (called "pre-contamination panoramas"); (ii) a set of panoramic images acquired after the scene has been possibly contaminated (called "post-contamination panoramas").It is assumed that the misalignment between corresponding panoramas acquired before and after the contamination, is limited, since images are captured from roughly the same position both times.A multi-step strategy is applied:  first, features are independently detected in all images and described through invariant descriptors.This task is performed using the the SIFT ++ implementation of SIFT algorithm (Vedaldi, 2010) provided within the MicMac open source suite of tools (MicMac).Specifically, SIFT ++ is a lightweight C ++ implementation of SIFT detector and descriptor, directly derived from the MATLAB/C implementation;  second, for each pair of corresponding pre-and postcontamination panoramas, the 2D positions of keypoints are mutually compared and a distance map is computed;  subsequently, keypoints extracted on changed objects are detected by a two-step approach: first, points featuring distances higher than 15 pixels are segmented; second, clusters including more than 100 features are detected by connected component labelling.Both thresholds have been experimentally determined, in order to cope with false differences due to small image misalignment affecting corresponding panoramas;  next, areas corresponding to the detected changed objects are masked on the pre-contamination panoramas, in order to limit the search space for the subsequent feature-based matching step.This takes as input the masked precontamination panoramas and DSRL images of evidence which has been, accidentally or intentionally, taken away from the scene and possibly found afterwards by investigators (called "evidence images").SIFT points detection, description and matching are performed using the algorithms available in MicMac, and provide for a filtered set of correspondences, after a RANSAC-based feature matching verification;  finally, these tie points are used to compute a geometric transformation that maps features of each evidence image to the panorama, that shows the highest number of correspondences with it.This feature-based alignment can be restricted to a planar parametric transformation that describes a projective 2D motion (homography) between the two sets of matched 2D points.Although a homographic model is not fully valid due to a lack of planarity of the captured scene, it represents here a useful and intuitive means to visualize the two images (pre-contamination panorama and evidence image) superimposed on each other and support the documentation task.

Developed methodology 2.2.1 Multivariate Alteration Detection (MAD) and Maximum Autocorrelation Factor (MAF)
Many techniques have been developed in the remote sensing domain, with the purpose of identifying the differences occurred on the Earth surface over time.This task, usually referred to as change detection, can be approached in a supervised or unsupervised way, the latter being preferred when no training samples or only little knowledge on the ground are available.Among the unsupervised techniques, the Multivariate Alteration Detection (MAD) is a broadly used mathematical analysis method of images linear transformation.Introduced by Nielsen (1998), MAD seeks to improve the simple image differentiating techniques by exploiting the Canonical Correlation Analysis (CCA).Indeed, the main principle is to make the images as similar (i.e.correlated) as possible, before computing their difference.The latter is carried out by using CCA to find two sets of linear combinations of the original variables, where the first two linear combinations (called canonical variates) are the ones featuring the largest correlation (called first canonical correlation).This process is then iterated to compute the higherorder canonical correlations/variates, under the condition to be orthogonal (i.e.uncorrelated) to the previous ones.If N is the maximum number of bands in first and second input images, the differences between the corresponding pairs of variates (called MAD variates or components), constitute N change maps, that are usually combined in a single multi-band image.
Since MAD analysis lacks in semantic interpretation, the adoption of a combined procedure can be preferred, to support the understanding of changes found by MAD.Nielsen proposed to apply the Maximum Autocorrelation Factor (MAF) transformation to the MAD components (Nielsen, 2010).
Indeed, MAF transform seeks to isolate the noise component of the data, by computing a new set of variates out of the original ones, where low-order components feature maximal spatial autocorrelation (signal), whereas the highest order variates feature minimal spatial autocorrelation (noise).Accordingly, the first MAF-MAD component will identify areas with maximum changes, while the noise is expected to be isolated in the lower order MAF-MAD components.
The use of MAD technique, either alone or in combination with MAF transform, is well-known in the remote sensing community (Coppin et al., 2002;Nori et al., 2009;Zanchetta and Bitelli, 2017).

MAD and MAF for contamination detection
The methodology developed to evaluate the performance of the proposed approach is summarized in Figure 2.

The Crime Scene House
The Crime Scene House (CSH) is a structure built at Staffordshire University (UK) to simulate a typical domestic home for crime scene investigation.It is a detached house with a large garden and off-road parking (Figure 3, left).It contains seven rooms that have been furnished to resemble a typical domestic home.There are also some permanent exhibits of crime scene investigation, e.g.examples of various types of bloodstain patterns.The extensive outside areas enable the simulation of outdoor scenes, such as clandestine burials, and vehicle crime scenes.Conceived as a training and learning facility for students and researchers, the main function of the CSH is to simulate investigations in a realistic setting.Various types of scenarios can be set up and processed, using the appropriate equipment and investigation protocols.In a way, such an environment provides for the perfect setting to determine the best results achievable with a particular investigation method.

Simulated crime scene scenario
The proposed methodology for the documentation of contaminated crime scenes is tested by setting up a mock environment at the CSH.Among its rooms, the "studio" is chosen for the experiments (Figure 3, right).The scene is characterized by traditional furniture and decorations.The walls and the ceiling have mostly textureless and bright surfaces.Furthermore, un-controlled and un-diffused illumination conditions are provided by a single big window.Several small objects are intentionally scattered into the environment, and a contamination event is afterwards simulated by taking away from the scene: • a gun replica, originally placed on the floor; • a bundle of banknotes, originally lying on the floor; • two cans, originally placed on a table and on a desk: • some flowers, originally lying on the floor.During the image acquisition phase, the Ricoh Theta panoramic camera is fixed on a tripod and remotely operated through a dedicated smartphone app.Particularly, a tilted acquisition setup is realized by rotating the camera horizontally (i.e., with a lens looking up and a lens looking down).This configuration allows the deformation of the objects in the equi-rectangular images to be reduced, since most of the evidence lies on horizontal surfaces, e.g.pavement and tables.Furthermore, since the room features several distributed pieces of furniture (e.g.tables, armchairs and bookcases) and the objects are randomly placed on them and on the floor, many partiallyoccluded areas should be dealt with.In order to cope with this issue, a camera network of sixteen panoramas is carefully planned in advance, resulting in a mean camera-to-camera distance of about 1.5 m.The same acquisition protocol is repeated for the post-contamination scenario.Although the camera positions (and orientations) during the second acquisition phase are planned to be as similar as possible to the previous ones, a small misalignment between corresponding panoramas is intentionally introduced, in order to simulate real documentation conditions.Indeed, after the scene is first recorded when the crime event is disclosed (corresponding to the pre-contamination scenario), the following postcontamination scenario will be potentially acquired by different operators, who will not exactly follow the same initial setup.Eventually, in both image acquisitions, the only adjustable camera parameters, namely ISO sensitivity and shutter speed, are set to 100 and 1/15 ms, respectively.Finally, assuming that single objects are found by investigators later on, and a reasonable suspicion exists that they were originally placed in the contaminated crime scene, a third image acquisition campaign is carried out.Single objects (i.e. the gun replica, the banknotes, the cans and the flowers) are imaged with a Nikon 3DX digital camera (6720 x 4480 pixels, 6 µm pixel size) equipped with a NIKON ED AF-S VR-Nikkor 70-200 mm 1:2.8 G lens, fixed at 105 mm by means of an insulating tape applied on the lens ring.The objects (see examples in Figure 4) are acquired at a mean distance of about 0.7 m, corresponding to a mean ground sample distance of 0.04 mm.All images are taken with the camera mounted on a tripod, after having placed the objects on a white surface to mask the background.

Results achieved with the developed methodology
Figure 5 shows two examples of results achieved by applying the proposed methodology (steps from 1 to 3 of Figure 1) to two couples of corresponding panoramas.Some remarks can be drawn as follows:  as expected (Figure 5, first and second row), keypoints are mainly extracted on surfaces featuring a significant texture (e.g.furniture, objects and floor) and along edges; almost no features are detected on homogeneous surfaces, such as the walls and the ceiling;  the colour-coded distance maps (Figure 5, third row) computed by comparing the positions of keypoints on preand post-contamination panoramas, clearly show which features have undergone any kind of change.Particularly, changes can be due to (i) different illumination conditions (resulting in sparse "changed" keypoints); (ii) small misalignments between the compared panoramas (resulting in clusters of "changed" keypoints); (iii) missing objects (resulting in clusters of "changed" keypoints);  by segmenting the biggest clusters of keypoints, featuring the highest position differences, the changed objects can be clearly detected (Figure 5, fourth row).Some false negatives (i.e.missing evidence) are found where objects look too small in the panorama (see the gun and the flowers, Figure 5 left), resulting in clusters smaller than the threshold.However, they are correctly identified by using other panoramas (see the gun and the flowers, Figure 5 right).Some false positives (i.e.false evidence) are detected due to image misalignments (see the edges, Figure 5 left), that generate big clusters of "changed" keypoints; however, they don't affect the subsequent step of feature matching between the masked panorama and the evidence images (see Figure 6).
Finally, Figure 6 shows three examples of results achieved by applying the proposed methodology (steps 4 and 5 of Figure 1) to two couples of panorama-evidence images.Two main observations stem from the achieved outcomes: (i) results confirm the changes detected by the proposed methodology (feature-based method), thus positively assessing its performance; (ii) results show that change detection algorithms, originally developed for satellite imagery, may be successfully applied also to forensic scenarios.Although based on a limited experimental dataset and setup, these outcomes offer a promising solution, bridging the gap between the remote sensing and the forensic research domains.

CONCLUSIONS
This research proposed a solution to the problem of documenting a crime scene, after a contamination event (either accidental or deliberate) has changed its original appearance.Indeed, crime scene contamination represents an open issue for police officers and forensic technicians.On the one hand, they can today exploit a wide variety of 3D image-based and rangebased modelling techniques, providing for a fast, reliable and accurate way to document and acquire reality-based data without hampering investigations.On the other hand, they need fast and automated tools to detect changes that occurred at the scene over time, and solutions to this problem are still limited.The contribution of the paper for addressing this need is twofold.First, a new methodology to exploit panoramic images and 2D feature-based methods (namely, SIFT detector and descriptor) was presented.The core idea is that SIFT features inherently contain scene information and, thanks to their good stability and invariance, can be exploited to detect possible changes that occurred at a scene surveyed with multi-temporal images.The proposed approach showed promising results when tested on a simulated crime scene scenario and contamination.In particular, it showed:  good performance in correctly detecting both the occurred changes due to contamination, and the original position of missing evidence.Indeed, all objects intentionally taken away from the scene were successfully identified and their location in the pre-contamination scene was retrieved;  high efficiency in terms of the time required for data acquisition and processing, and equipment needed for imagery capture.Indeed, only four couples, out of the original sixteen panoramas pairs, were actually used for detecting the changed objects and their original positions.The data were acquired with a low-cost and user-friendly panoramic camera, and the entire processing chain took place only in the 2D space, i.e. no 3D reconstruction was needed;  a simple and intuitive approach for visualizing the results.
Indeed, the original location of evidence is visualized by mapping the object on its original position in the precontaminated panorama, or through homologous rays.
Second, the paper demonstrated the applicability of change detection algorithms, originally developed within the remote sensing domain, to process RGB panoramas for forensic applications.MAF/MAD methods were successfully applied to detect the missing objects, thereby confirming results achieved with the proposed feature-based pipeline.
In future works, it would be interesting to test the performance of the feature-based, change detection approach, when dealing with challenges, like (i) panoramas featuring significantly different illumination conditions, (ii) missing evidence of a smaller size, compared to the objects used in these experiments, and (iii) increased misalignment between corresponding panoramas.In these cases, the adoption of a combined procedure may be a successful solution, by first exploiting MAF/MAD to identify the changes, and then applying the proposed feature-based method to detect the original position of contaminated evidence.

Figure 2 .
Figure 2. The developed methodology.Each couple of corresponding pre-and post-contamination panoramas are processed using the open-source software Orfeo ToolBox (OTB), a remote sensing, image-processing library developed by CNES, the French Space Agency.A three-step strategy is adopted:  first, image co-registration is refined using an iterative approach based on cross-correlation (CC), to estimate a best match between local patches.The algorithm computes a three-band disparity map between the two images, that contains the offsets along x and y, and the CC values.This map is then used for image warping;  second, change detection between the registered panoramas is performed by adopting the MAD algorithm.A RGB MAD map is thus produced, consisting of three bands that represent the variates (change maps) sorted by increasing correlation;  finally, the MAF transform is applied to the MAD variates, and the lowest order MAF-MAD component (i.e. the first component) is analysed to detect the changes (contamination) that occurred.

Figure 3 .
Figure 3.The Crime Scene House: an overview of the building (left) and the "studio" room used as experimental set up (right).

Figure 4 .
Figure 4. Examples of evidence images: banknote (left) and gun replica, before and after the application of the Wallis filter (centre and right, respectively).

Figure 5 .
Figure 5. Feature-based contamination detection.Two examples (left and right) showing: keypoints on pre-contamination panorama (first row), keypoints on corresponding post-contamination panorama (second row), change detection map (third row; distance ranging from 0 pixels-blue to more than 40pixels-yellow), changed objects (fourth row).
First (Figure6, up), tie points and homologous rays computed by feature-based matching, clearly show the correspondences detected between the masked panorama (in blue) and the evidence image (in orange).This can help investigators in highlighting the original position of the evidence on the precontaminated crime scene.Second (Figure6, centre), the homography estimation can be performed in SEL, a tool of the MicMac suite.Given the set of matched feature points, SEL computes the eight-parameter homography matrix through a least mean square optimization (L2 minimization).Optionally, it displays the two images superimposed on each other, after applying the computed homography transformation that maps the second image (i.e. the evidence image) to the first one (i.e. the panorama image).This intuitive visualization may further support the documentation of the contaminated crime scene.Finally, in the case of low-textured objects, like the gun replica (Figure4, centre), an advanced image content enhancement technique is applied before computing the tie points.It is a modified version of the well-known adaptive Wallis Filtering algorithm(Wallis, 1976), as implemented in the open-source photogrammetric tool GRAPHOS(González-Aguilera et al.,  2018).Thanks to this pre-processing, performed on the masked panorama and the evidence image (Figure4, right), three correspondences are detected by feature-based matching (Figure6, bottom).Although they are not enough to compute the eightparameter homography matrix, homologous rays make the identification of the gun's original position, an easy and intuitive task.

Figure 6 .
Figure 6.Feature-based detection of the original evidence position: homologous rays and homography map between the masked panorama and the banknote (up and centre); homologous rays between the pre-processed masked panorama and gun image (bottom).4.2 Results achieved with the proposed methodology Examples of results achieved by applying the reference methodology (change detection) are shown in Figure 7 and 8.For the sake of comparison, they correspond to the cases presented in Figure 5 left and right, respectively.On the top, the RGB MAD map is reported: it represents the three-band map derived by computing the three change maps (variates) from the original pre-and post-contamination panoramas.Although some areas are well distinguished from the surroundings, the semantic interpretation of the output is not trivial and univocal.To help in understanding the meaning of the change found by MAD, the MAF transformation of the MAD variates is a good solution, confirming what is well-known in the remote sensing literature.Indeed, the lowest order component of the MAF transform applied to MAD variates (MAF-MAD component 1, Figure 7 and 8, bottom), clearly highlights the changes occurred between the pre-and the post-contamination panoramas.In particular, all three changed objects visible in the two panoramas (i.e. the gun replica, the banknote and the flowers) are successfully detected in the MAF-MAD component 1 of Figure 7 and Figure 8.

Figure 7 .
Figure 7. MAD and MAF for contamination detection.Results of the reference methodology (corresponding to Figure 5, left), showing: RGB MAD map (up) and MAF-MAD component 1 (bottom).

Figure 8 .
Figure 8. MAD and MAF for contamination detection.Result of the reference methodology (corresponding to Figure 5, right), showing: RGB MAD map (up) and MAF-MAD component 1 (bottom).

Weight (g) 102 Sensor size (mm) 6
Panoramic images are acquired with the Ricoh Theta SC 360 Camera.It is a cheap and compact spherical panoramic camera, featuring a dual-lens configuration which covers the entire scene by 360 x 180 degrees.The camera is equipped with a 12 Megapixel CMOS optical sensor, with 6.17 x 4.55 mm size.It features a fixed aperture (f/2.0) and fixed focal length (1.3 mm, 7.3mm 35mm-equivalent) lens.The single images are stitched together using embedded software and there is no possibility to access the raw pre-stitching shots in the current version of camera.Indeed, images are currently exported in equirectangular JPEG format (5376 x 2688 pixels).The main technical specifications of the camera are listed in Table1.Dimensions (mm)45 (W) x 131 (H) x 23 (D) .17x 4.