3D MODELLING AND RAPID PROTOTYPING FOR CARDIOVASCULAR SURGICAL PLANNING – TWO CASE STUDIES

: In the last years, cardiovascular diagnosis, surgical planning and intervention have taken advantages from 3D modelling and rapid prototyping techniques. The starting data for the whole process is represented by medical imagery, in particular, but not exclusively, computed tomography (CT) or multi-slice CT (MCT) and magnetic resonance imaging (MRI). On the medical imagery, regions of interest, i.e. heart chambers, valves, aorta, coronary vessels, etc., are segmented and converted into 3D models, which can be finally converted in physical replicas through 3D printing procedure. In this work, an overview on modern approaches for automatic and semi-automatic segmentation of medical imagery for 3D surface model generation is provided. The issue of accuracy check of surface models is also addressed, together with the critical aspects of converting digital models into physical replicas through 3D printing techniques. A patient-specific 3D modelling and printing procedure (Figure 1), for surgical planning in case of complex heart diseases was developed. The procedure was applied to two case studies, for which MCT scans of the chest are available. In the article, a detailed description on the implemented patient-specific modelling procedure is provided, along with a general discussion on the potentiality and future developments of personalized 3D modelling and printing for surgical planning and surgeons practice.


INTRODUCTION
According to the World Health Organization (WHO), cardiovascular diseases (CVDs) are the first cause of human death. They include all the diseases of the heart and circulation including coronary heart disease, angina, heart attack, congenital heart disease (CHD) and stroke. The incidence of CVDs can be reduced through prevention by controlling behavioural risk factors, such are unhealthy diet, physical inactivity, tobacco use and alcohol abuse. Nevertheless, surgery still represents the ultimate solution for defects or abnormality of the heart and blood vessels that can be present from birth (congenital) or arise because of age. In the meantime, population ageing is generally growing, thanks to the improving life conditions (better nutrition, health care, economic well-being). This translates in a health care system that has to deal with a constantly increasing number of potential patients and, consequently, needs be more efficient.
In the last years, diagnosis, surgical planning and intervention have taken advantages from 3D modelling and rapid prototyping techniques. The procedure starts from the collection of medical imagery, usually in the form of volumetric data that consists in sequentially 2D slices and represents the internal structures of human body. Parts of interest (tissues, organs, etc.) are segmented and transformed in 3D models that can be physically reproduced through rapid prototyping procedures. The paper discusses all the steps of the outlined procedures, presenting a brief introduction on medical imaging and CT technology (Section 2), with a detailed review of the existing methods for medical imagery analysis and processing Sections 3,4 and 5). An overview about 3D printing technology adopted for medical applications is also provided (Section 6). Moreover, two case studies are presented (Section 7): a patient-specific procedure has been adopted with the aim of producing physical replicas for surgical planning in case of complex heart diseases.

MEDICAL IMAGING
Medical imaging includes different imaging modalities and processes that aim at achieving and displaying information about the interior of human body for diagnostic and treatment purposes. Over the years, a plenty of different medical imaging modalities has been developed, using different technology and creating different kinds of images. Common types of medical imaging include: (i) X-ray (radiography, computed tomography -CT, etc.), (ii) magnetic resonance imaging (MRI), (iii) ultrasound and (iv) nuclear medicine (e.g., positron emission tomography -PET). Each of them has its own advantages and disadvantages, both for potential patient's health hazard (some of them require exposure to radiation) and information gathered and displayed. For example, CT and MRI are sensitive to different tissues properties, so that soft tissues are poorly visible in CT images, while they are distinctly evident in MRI. CT, PET, MRI provide 3D information in the form of thin crosssectional "slices" of the entire body or parts of interest, and are therefore called tomography imagery. The slices are provided in the form of digital pictures stacked through the measurement volume, where each pixel represents the spatial position of the imaged element and its colour (grey or RGB values) the reaction of the tissue to radiation (X-rays or gamma-rays) or magnetic field, in case of CT, PET and MRI respectively. Medical imagery usually provide data organised in a Cartesian rectilinear grid, which can be either isotropic or anisotropic, depending whether the intra-slice distance is equal to the in-slice resolution or not. In CT scan, an X-ray source, a tube, produces an X-ray beam that passes through the patient body. Some photons are then absorbed, other scattered. The attenuated X-rays are collected by X-ray detectors on the opposite side and converted into digital data, collected into square digital images. The value of each pixel in a CT scan represents the attenuation value or reduction of X-ray transmission, which depends on the atomic composition and density of the crossed tissue. The attenuation values are expressed in Hounsfield units (HU) that represent the attenuation measurement relative to water (Equation 1). The Hounsfield scale is a linear scale, ranging from -1024 to +3071 HU. The value of each element of the 2D scan is computed as: where water and air are respectively the linear attenuation of water and air. The attenuation coefficient of air is nearly zero, so that a change of one Hounsfield unit (HU) can be interpreted as a change of 0.1% of the attenuation coefficient of water. Figure  2 shows typical HU values for different tissues

Cardiac CT
In cardiac CT, there are two imaging procedures: coronary calcium scoring with non-contrast CT and non-invasive imaging of coronary arteries with contrast-enhanced CT.
Typically, non-contrast CT imaging exploits the natural density of tissues. As a result, various densities using different attenuation values such as air, calcium, fat, and soft tissues can be easily distinguished. In comparison, contrast-enhanced CT is used for imaging of coronary arteries with contrast material such as a continuous infusion of a high concentration of iodinated contrast material (Schoepf, 2007). Especially in the recent rapid advances in CT technology, CT can provide detailed anatomical information of chambers, vessels, coronary arteries and coronary calcium scoring. Additionally, CT imaging provides functional as well as anatomical information, which can be used for quantitative assessment for wall tickening and regional wall motion (Kristensen et al., 2009).

DICOM format
Medical imagery are commonly stored in a standard format file, defined by the National Electrical Manufacturers Association (NEMA) known as "Digital Imaging and Communications in Medicine" (DICOM).
Similarly to the Exif tags for digital photographs, DICOM files consists of a number of attributes that contain: (i) patient related information (name, sex, age, etc.), (ii) modality and imaging procedure information (type of machinery, acquisition parameters, radiation dose, contrast media, etc.) and (iii) image related information (image dimension, in-plane resolution or pixel size, slice thickness, slice distance, etc.).

IMAGE ENHANCEMENT AND RESTORATION
Digital medical imagery often suffers from different kinds of degradation effects, such as artefacts due to patient motion or interferences, poor contrast, noise and blur ( Figure 3).
To improve the quality and visual appearance of medical images, two main procedures are usually adopted, namely image restoration and image enhancement (Al-Almeen et al., 2013). Image restoration algorithms primarily aim at reducing blur and noise in the processed image, naturally related to and introduced by the data acquisition process. Image enhancement techniques are mainly devoted to contrast enhancement, in order to extract or accentuate certain image features. The aim of the enhancement process is to improve the understanding of information content and obtain an image more suitable than the original for automated image processing, such as, for example in the case of medical imagery, for highlighting structures (tissues, organs, etc.) of interest. Range compression, contrast stretching, histogram equalization with gamma correction are usually adopted to enhance the quality of medical images (Preethi & Rajeswari, 2010). In general, despite the effectiveness of each single approach, usually a combination of different methods allows to achieve the most effective image enhancement result (Maini & Aggarwal, 2010).

SEGMENTATION AND CLASSIFICATION
Image segmentation is the process of dividing an image into regions with similar properties such as grey level, colour, texture, brightness and contrast (Sharma & Aggarwal, 2010). In medical imagery, the role of segmentation consists in identifying and subdividing different anatomical structures or regions of interest (ROI) in the images. As result of the segmentation task, the pixels in the image are partitioned in non-overlapping regions, belonging to the same tissue class. Some approaches use texture content to perform image segmentation and classification: the aim of texture based segmentation method is to subdivide the image into region having different texture properties, while in classification the aim is to classify the regions which have already a) b) c) Figure 3. Examples of medical images affected by motion artefacts (a), interferences due to an implanted pacemaker (b) and poor contrast (c). been segmented (Sharma & Aggarwal, 2010). Labelling is the process of assigning a meaningful description or annotation to each segmented region based on its features, for example healthy or tumorous tissue (Pham et al., 2000). Automatic segmentation of medical images is still a difficult task as medical images are complex in nature and often affect by intrinsic issues: i. partial volume effects, i.e. artefacts occurring when different tissue types mix up together in a single pixel and resulting in non-sharp boundaries; ii. intensity inhomogeneity of a single tissue class that varies gradually in the image, producing a shading effect; iii. presence of artefacts (Figure 3a and Figure 3b); iv. similarity of grey values for different tissues. Classifications of existing segmentation methods have been attempted by several authors (see for example: Pham et al., 2000;Sharma & Aggarwal, 2010;Withey & Koles, 2008). Similarly to other image analysis fields also in medical image segmentation, automatic methods are classified as supervised and unsupervised, where the main difference resides in the operator interaction required by the first approaches throughout the segmentation process. The methods that identifies regions of interest by labelling all pixels/voxels in the images/volume are known as volume identification methods. On the contrary, approaches that recognises the boundaries of the different regions are called boundary identification methods (Withey & Koles, 2008). Low-level techniques usually relay on simple criteria based on grey intensity values, such as thresholding, region growing, edge detection, etc. More complex approaches introduce uncertainty models and optimization methods, like statistical pattern recognition (Markov random field -MRF, Bayesian classifiers, etc.), deformable models, graph search, artificial neural networks, etc. Finally, the most advanced methods may incorporate higher-level knowledge, such as a-priori information, expert-defined rules, and models. Methods like atlas-based segmentation and deformable model belong to this last group. For patient-specific application in surgical planning a fully and accurate automatic segmentation approach would be desirable to make the process fast and reliable. Unfortunately, anatomical variability and intrinsic image issues limit the reliability of fully automatic approaches. At the end of the segmentation process, the operator interaction is still required for error corrections (Olabarriaga & Smeulders, 2001). Interactive segmentation methods, employing for example manual segmentations in a small set of slices and automatic classification of the remaining volume using patch-based approach (Pace et al., 2015), provide promising results.

Segmentation in cardiac imagery
Various segmentation techniques have been proposed for cardiac CT applications. As pointed out in Kirişli et al. (2012), most of the studies realised in the last years focused on the left ventricle (LV), because functional parameters derived from the diastolic and systolic LV volume are important predictors for heart disease (Adabi et al., 2010). Nevertheless, automatic segmentation and modelling of four cardiac chambers is a fascinating and promising research field (Lorenz & von Berg, 2006). A whole total heart model allows for the diagnosis of diseases also in the S other chambers, such as left atrial fibrillation and right ventricle (RV) overload. Funka-Lea et al. (2006) proposed a method to segment the entire heart using graph-cuts. Segmenting the entire heart was performed for clearer visualization of coronary vessels on the surface of the heart. They attempted to set up an initialization process to find seed regions automatically using a blowing balloon that measures the maximum heart volume and added an extra constraint with a blob energy term to the original graph-cuts formulation. Jolly (2006) prosed a method for extracting the myocardium in 4-D cardiac MR and CT images using the graphcuts as well as EM-based segmentation. Zheng et al. (2008) presented a segmentation method based on the marginal space learning by searching for the optimal smooth surface. Modelbased techniques were also adopted for cardiac CT image segmentation using active shape models with PCA (Ecabert et al., 2006), atlas-based segmentation (Išgum et al, 2008;Kirişli et al., 2012), machine learning (Zheng et al., 2008), methods for region growing (Dey et al., 2008;Mühlenbruch et al., 2006) and thresholding (Juergens et al., 2008;Yalamanchili et al., 2010) were also employed.

VISUALIZATION, RENDERING AND RECONSTRUCTION
Different methods are available to display and reconstruct 3D volumetric data on a 2D screen. All the methods can be broadly classified into two main approaches, namely 2D and 3D display techniques (Robb, 2003). 3D display methods can be further divided into two branches: (i) 3D visualization and rendering (Figure 4a), (ii) surface ( Figure 4b) and volume meshing. The first class of techniques aim at interactively visualizing and rendering volumetric data in 2D displays in an intelligible manner, practically useful in everyday diagnostics (Perandini at el., 2010). On the contrary, the second approaches can produce models that are suitable for computational modelling and 3D printing. The rendering procedure is composed of two processes, namely viewing or projection and shading (Meißner et al., 2000). The projection process determines, for each screen pixel, which objects are seen by the viewing ray starting from the pixel into the scene. Shading or illumination is the technique to enhance the appearance of a geometric object, modelling effects like shadows, light scattering and absorption. Classification and segmentation of internal structures (Section 4) can be used to assign to each element properties such as colours, opacity and other material properties, adopted for effective rendering. Similarly, segmentation results are the starting point for surface models generation, using usually marching cube algorithm (Lorensen & Cline, 1987). The resultant surface can be used as the starting point for either generation of higher order computer-aided design (CAD) representation, such as nonuniform rational B-splines NURBS-based surfaces, or for meshing improvement using, for example, advancing front (AF), Delaunay meshing techniques (Young et al., 2008) or Poisson surface reconstruction method (Lobos & Rojas-Moraleda, 2013). Surface meshes usually require some editing to satisfy requirements of water-tightness (no holes) and smoothness. From surface polygonal meshes (Figure 4b), volumetric meshes can be derived. Unlike polygon meshes, which represent only the surface as polygons, volumetric meshes also represent the interior structure of the object and are characterised by hexahedral and tetrahedral polygons. Volumetric meshes are specific for finite element analysis (FEA) and computational fluid dynamics (CFD) studies, such as the case of cardiac computational modelling (Lopez-Perez et al., 2015).

Surface model editing and preparation for 3D printing
Topological correction, decimation, Laplacian smoothing, and local smoothing are usually needed to make 3D surface model suitable for 3D printing. Currently, image-processing software applications are not able to generate 3D models directly applicable to 3D printing. Therefore, the post-processing of 3D surface models (i.e., correction of erroneous structures and elimination of artifacts) relies heavily on the expert clinical and anatomic knowledge. Distinguishing between abnormal but real patient structures and image artifact is often difficult and automatic approaches may fail. In efforts to overcome this limitation, a specific image-processing and 3D reconstruction workflow has been proposed by Kim et al. (2008). This approach, to be used during data processing, has the aim of eliminating much of the subjective "expert driven" workflow in deciphering between true anatomy and imaging artifact, ultimately resulting in the generation of more anatomically accurate digital 3D surface models, and subsequently physical models, in a shorter amount of time.

3D PRINTING
In the last year, 3D printing is becoming an emerging research field in medicine in general, and in cardiovascular applications in particular (Jacobs et al., 2007;Shiraishi et al., 2010;Schmauss et al., 2014). The 3D printing technology used in medicine is defined as either subtractive or additive depending on the printing method used. The only subtractive technique used in medical applications is milling, in which the shape of the model is milled from a block of material (either polyurethane or other foam product). The most common additive technologies used in medicine are selective laser sintering, fused deposition modelling, multijet modelling, and stereo lithography. Selective laser sintering uses a high-power laser to fuse small particles of plastic, metal, or ceramic powders into a 3D object. The laser selectively fuses a pattern into the surface of a thermoplastic powder bed by scanning cross sections generated from the CAD 3D surface model. After a cross section is scanned and laser fused, the powder bed is lowered by one layer of thickness, a new layer of thermoplastic powder is applied on top, and the process is repeated. In fused deposition modelling, a plastic filament is forced through a heated extrusion nozzle that melts the filament, moves in both a horizontal and vertical direction as guided by the CAD software application, and deposits a layer of material that hardens immediately on extrusion. Multijet modelling essentially works like a normal inkjet printer but in 3D space. In this process, layers of fine powder (plaster or resins) are selectively bonded by printing a waterbased adhesive from the ink-jet printhead in the shape of each cross section as determined by the CAD file. Each layer quickly hardens, and the process is repeated until the model is complete.
In stereo lithography, models are built through layer-by layer polymerization of a photosensitive resin. A controlled laser generates an UV beam that draws on the surface of a pool of resin stimulating the instantaneous local polymerization of the liquid resin in the outlined pattern. A movable platform lowers the newly formed layer, thereby exposing a new layer of photosensitive resin, and the process is repeated until the model is complete.
Depending on their intended application (i.e., education, catheter navigation, device sizing and testing, and so on), physical models may be printed in multiple materials (e.g., plastic, rubber polymers, etc.).

CASE STUDIES
In the present section, two real case studies are presented. The two patients suffer from different heart diseases: an old woman with a vascular prosthesis in the descending part of the aorta and a young man with severe pulmonary regurgitation and right ventricular dilatation. In both the case, a patient-specific modelling procedure was adopted, whose main steps are identified in Figure 1.

The implemented patient-specific modelling procedure
The multi-slice CT (MCT) data of the chest available for the two patients are processed in the open-source medical image computing software package 3D Slicer (https://www.slicer.org The first step of the followed patient-specific approach entailed a denoising process, involving a Gauss blur filter and histogram stretching procedure applied to the MCT volumetric data. While in the aorta modelling only one class was needed, in the second case study the volumetric data is segmented into three different classes: (1) chest bones, (2) pulmonary root and (3) whole heart including the ascending aorta trunk. In both the examples, the segmentation task was performed employing a semi-automatic procedure consisting in the following steps: (i) grown cut, (ii) thresholding, (ii) region growing and eroding, (iii) disconnected components removal, (iv) holes filling. Binary masks were created, one for each relevant class.
The following step consisted in the generation of the separate surface mesh models using MC and smoothing algorithms. The surface mesh models were exported and the remaining topological errors (small holes, self-intersecting triangles, small tunnels) are corrected in specialized surface modelling tool. In the presented case studies commercial software packages were employed, namely Polyworks by Innovmetric (http://www.innovmetric.com/) and Geomagic Studio by 3D Systems (http://www.geomagic.com/en/). Alternatively, similar results are obtainable using free, open-source tool like MeshLab (http://meshlab.sourceforge.net/) and Blender (https://www.blender.org/). The triangular mesh models were then optimized, in order to reduce the number of polygons, at the same time keeping the level of details and resolution of the original model. Stereolithographic (STL) models were generated and a procedure for accuracy assessment of the obtained model is devolved. Briefly, sections in the three orthogonal main plane (i.e. axial, sagittal and coronal) were extracted from the mesh models at the same position of the original MCT slices. In the obtained sections several distances were measured and compared to the same measurements taken in the corresponding original sections. With this method, the measurements taken in the threedimensional models were found to differ by at most 5 mm from the same measurements extrapolated from the original MCT slices. A further verification was performed by re-importing the STL mesh models in 3D slicer in order to check the overlapping with the volumetric data. The last step of the procedure required the printing of the verified STL mesh model using rapid prototyping procedure. The final products were 3D patient specific physical models used by the surgeons for surgery planning.

Aorta modelling
The 79 years old female patient suffered for sever aortic regurgitation and presented a quite severe clinical picture, including hypertension, diabetes, epilepsy, etc. She had previously undergone surgery for an aneurism involving the ascending aorta and the brachiocephalic trunk was re-implanted in the ascending aorta (Gallo et al., 2016). The surgery team decided to implant a surgical trans-catheter stent-valve (CoreValve Evolut R29mm, http://www.corevalve.com/us/index.htm), using the transfemoral route. A MCT was performed to provide high-resolution images of the aorta in order to evaluate the anatomic relationship between the brachiocephalic trunk and the aortic annulus. Following the procedure described in Section 7.1, the internal wall of aorta vessel was segmented in the volumetric data and the surface mesh model generated. A constant thickness was applied outwards and three reference points are represented as 3mm diameter spheres (Figure 5a). The points materialized on the mesh model the nadir (hinge point) of each aortic cusp. Those three reference points, when virtually connected, formed the "virtual basal ring" of aortic valve, used by the surgeon as repair for the deployment of the trans-catheter stent-valve. The STL model was re-imported in 3D Slicer for accuracy assessment ( Figure 8b) and finally printed in transparent material (polylactic acid -PLA, Figure 8c) by an Italian TalentLab (help3d.it) with a Sharebot NG printer (http://www.sharebot.it/?lang=end).

Heart, chest bones and pulmonary root
The 42 years old male patient presented a serious CHD, known as pentalogy of Fallot, involving many anatomical abnormalities of the heart. During his life, he had undergone to a number of surgeries, trying to cope with the diseases, but in the last years he had developed severe pulmonary regurgitation with right ventricular dilatation. Consequently, it was planned to replace his right ventricular outflow tract (RVOT), i.e. the extension of the ventricular cavity which connects to the pulmonary artery, with a decellularized cryopreserved homografts (DCHs). DCHs are heart valves from human donor which have been modified via tissue engineering. As for the previous patient, a MCT was performed to provide high-resolution images of whole heart, chest bones, pulmonary root and valve with clear localization of anatomical relationships. Three separate surface mesh model were generated from the corresponding binary masks, i.e. chest bones, pulmonary root and whole heart including the ascending aorta trunk ( Figure 6 a and  b). The accuracy assessment was performed by comparing reference measurements taken on the original MCT slices ( Figure  6 d) and on corresponding sections extracted from the mesh models ( Figure 6 e and f). The models were printed in PLA using a Sharebot printer and the obtained physical replicas (Figure 6c) are used for the surgery planning.

CONCLUSIONS
Medical imagery is a fundamental tool in modern medicine; its quality is constantly improving and, at the same time, methods for analysis, understanding and interpretation are progressing. Moreover, 3D modelling and rapid prototyping are emerging technologies for medical diagnosis, surgical planning and intervention and training. They allow for patient-specific approaches, helping in finding the best solution also in complicate and peculiar cases. In the present work, an overview on medical imaging techniques, along with the fundamental steps for 3D modelling and printing were presented. A patient-specific approach was developed based on open source tool for medical image computing. The procedure was applied for two patients, suffering from severe heart diseases. In both the cases, physical replicas were realized through 3D printing for surgery planning. The study shows that semi-automatic approaches for medical imaging 3D processing have reached promising results, but intervention of experts is still necessary.