PROBABILISTIC RECONSTRUCTION OF ORTHODOX CHURCHES FROM PRECISION POINT CLOUDS USING BAYESIAN NETWORKS AND CELLULAR AUTOMATA

: The point cloud interpretation and reconstruction of 3d-buildings from point clouds has already been treated for a few decades. There are many articles which consider the different methods and workows of the automatic detection and reconstruction of geometrical objects from point clouds. Each method is suitable for the special geometry type of object or sensor. General approaches are rare. In our work we present an algorithm which develops the optimal process sequence of the automatic search, detection and reconstruction of buildings and building components from a point cloud. It can be used for the detection of the set of geometric objects to be reconstructed, independent of its destruction. In a simulated example we reconstruct a complete Russian-orthodox church starting from the set of detected structural components and reconstruct missing components with high probability


INTRODUCTION 1.1 Motivation
The development of science, technology and equipment as well as the human ability to invent new constructions has allowed for the creation of different unique objects in the real world.In the course of time, a lot of information about former cultural objects got lost, some objects were strongly damaged or ruined.The digital reconstruction makes the understanding of building principles easier, especially for partly destroyed or no more existing objects.
The reconstruction of lost historical and architectural objects is an up-to-date topic in many researches.With 3D laserscanning we are able to create the 3D construction plans of real scenes with high precision and completeness.Complex geometry reconstruction from point clouds in the context of the big data problem is not a trivial duty and needs methods and algorithms for the optimization of data processing.Due to the large amount of data, the choice of efcient methods is an important task.
As a standard rule, object reconstruction begins with feature detection, in case of building reconstruction with geometrical features.The existing methods are mostly appropriate for simple objects such as recognition of planes or spheres.For complex objects, combining different geometric entities, the choice of one particular detection method is not always appropriate because each method offers its own advantages for specic types of geometric entities.
In this article, we develop the mathematical model of a new method, which allows an optimized extraction of geometrical information from laserscanning point clouds and its efcient interpretation for further reconstruction.This method is suitable for object reconstruction from incomplete data which can be the result of missing object parts.This work is carried out in the context of recent research in virtual reconstruction of destroyed orthodox churches, which are known by their complex architecture.

Previous works
Considering previous work, we focus two aspects: 1. point cloud interpretation and 2. reconstruction from precision point clouds.
In our case, point cloud interpretation means extraction of analytical, geometrical and semantical information from point clouds.
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W3, 2017 3D Virtual Reconstruction and Visualization of Complex Architectures, 1-3 March 2017, Nafplio, Greece This contribution has been peer-reviewed.doi:10.5194/isprs-archives-XLII-2-W3-187-2017 Even though these methods are quite robust, some of them are dependent on the number of processing iterations, need start sampling and are not always correct for complex objects, which can be a combination of different geometries.In case of complex architecture, which cannot be approximated only with geometrical primitives, the detection method of one geometric object can differ from others.Chosen method determines further reconstruction of the whole object.
There are different reconstruction approaches based on interpreted information from point clouds.Some reconstruction methods use a strict prototype model.In many articles about reconstruction of cultural heritage objects a library of typological architectural elements taking in account construction canons is used.
In (Quattrini et al., 2015) a destroyed architectural object has been completely reconstructed from the TLS point cloud as well as single classied archaeological samples according to practical and theoretical canons of roman architecture.(Dore and Murphy, 2013) generated digital historical models using Historic Building Information Modelling (HBIM) containing parametric library objects and procedural modelling techniques.Further on, ( (Dore et al., 2015)) have developed a set of rules and algorithms for the automatic combination of parametric library objects and generation of HBIMs from survey data (historic surveys and recent laser scan survey as segmented point cloud and cut sections).A conceptual framework is based on the denition of shape grammar, which allows for the automatic generation of 2D and 3D geometries from a basic vocabulary of shapes.The reconstruction is supported by architectural rules and proportions.There are some program applications allowing for a semi-automatic modeling of architectural forms according to library of structural elements, in which the model parameters of structure element are estimated from user dened keypoints ( (Kivilcim and Duran, 2016)).(Huang et al., 2011) developed roof decomposition rules for reconstruction of LoD2 buildings.Based on a predefined library of primitives a generative modeling has been conducted to construct the target roof that fits the data.Extracted primitives from a point cloud were composed and merged.(Nguatem et al., 2013) extracted a ridgeline from the highest points of the point cloud, which was bounded with ground plan, for roof model tting using likelihood principle.
The reconstruction of single architectural models is considered in (Canciani et al., 2013).The method is based on the extrusion path modeling of architectural elements from point cloud section, which have been compared with a knowledge based model.
In many articles, a building reconstruction is based on integration of extracted geometrical information about single object parts according to different composition rules.
Extracted straight lines and planes (e.g. using RANSAC, Hough transform) solve often for planar object reconstruction and decomposition of its elements (e.g.roofs - (Nizar et al., 2006), (Arefi et al., 2010), buildings -(Rusu et al., 2009)).(Verma et al., 2006) detected planes and rectangular outlines for roof composition using Roof Topology Graph.(Kada and Wichmann, 2013) generated complex building shapes using Boolean intersection of half-spaces, which dene convex building components.(Xiong et al., 2015) represented roofs with topological graphs and applied the Minimum Cycles Method for roof decomposition using extracted geometrical primitives from airborne LiDAR data.
In the last years, probabilistic approaches are widely used in recognition processes.Probabilistic graphical models (PGM), developed by (Koller and Friedman, 2009), integrate schematic graphical object representation (e.g. as graph) with different stochastic statistic models, like Hidden Markov Model (HMM), Conditional Random Field (CRF) and Bayesian nets, allowing for probabilistic decision-making.(Ruiz-Sarmiento et al., 2015) integrated PGMs as Conditional Random Fields (CRF) with Semantical Knowledges (SK) for representation of object relations in context of input scene.(Xiong and Huber, 2010) extracted and classied planar regions for further object recognition using CRF.In (Anand et al., 2013) used an isomorphic to MRF (Markov Random Fields) model for object recognition and classication in certain scene (e.g.ofce, house).(Förstner, 2013)showed the efciency of object parameter estimation and classication, which are optimized through implementation of Bayesian nets and MRF in context of exible construction of graphical models.
In all cases, a big data problem is relevant for point cloud processing.The role of high mechanism complexity and the creation of self-reproducing intelligent automates for acceleration of technological processes has been discussed already for many years (e.g.(v. Neumann, 1966)).An additional example from such automates is a cellular automaton (CA), which can be based on varies rules (using for example Bayes theory and Bayesian networks ( (Wolfram, 1983), (Neapolitan, 2004))).
This way is favored in our actual work.

Outline of algorithm
The developed method is based on a probabilistic approach and discrete mathematics methods, namely Bayesian networks and cellular automata theory.Let us consider an algorithm which makes an automatic choice of an acceptable method for detection of different geometrical entities in a complex object and its further automatic reconstruction from incomplete data.Thus, we consider a-priori known object conguration with its pre-dened semantic information, which is organized like a graph.A combination of Bayesian nets with cellular automata allows to dene an optimal • geometric candidate for object detection, • recognition way, • decision-making for reconstruction from incomplete data.

Research object
Our research objects are stone orthodox churches.There is the large number of such churches in the territory of Russia, Belarus and Ukraine.Several churches have been destroyed and are not used as religious institution any more.There is a huge interest in the restoration of these objects.
It is necessary to observe the number of canons, determined by religion at the churches construction.These canons resulted in the evolution of the culture in that territory (and that time), in which we observe this or that church.Nevertheless it is possible to claim that each church has exactly the certain topological structure of some elements determined by canons.Each of these elements are geometrically characterized by its properties.

Knowledge representation
We model our objects -orthodox churches -with colored-oriented topological graphs G = (V, E).The vertices V are the structural components of the church, like a cupola or a crucix with arbitrary geometries.The edges E are the neighboring relations between the elements of V, which are attributed by conditional probabilities to the edges.The objects differ in their complexity (cf.Fig. 1).
We take in account the following encoding for the representation of some components: • The letters in the vertices are the specic structural components with a specic geometry and localization.
• The edges, coded with a line, show the topological and probabilistic connection or the probabilistic inuence of the components on each other (for example: if one component was detected, then we can claim, with a dened probability, that another component must be detected or reconstructed and vice versa).
• For the visual comfort we represent the different geometry types with different colors, i.e. each class of detected geometry type has its own color.
A database of certain amount of churches with classication of structural elements was built for derivation of probabilistic relations between elements and solves as a base for further construction of Bayesian network.In our case, the abbreviations of the church parts are: Kr -crucifix, HK -main cupola, T r -cylinder, St -prop, D -roof, X -nave, OAr -sacrifice altar right, Dr -sacrifice altar roof (right), HA -main altar; D -main altar roof, OA l -sacrifice altar left, D l -sacrifice altar roof (left).

Mathematical background
Let us consider a mathematical method solving a correct decision problem.This means in the context of our research, that we have checked the information about some parts of our object, we can apply the arbitrary method of recognition and search an object in a certain place.Consider a method based on the use of Bayesian networks as one method to guide this search process.
We use the basic formula (Bayesian Theorem) P (θ|I) = P (θ)P (I|θ) where • Iis the information about those parts of object, which were detected; • P (I) is the probability of I, dened as the frequency of objects occurs as identied as I in all the set of the churches; • θ are the identiers of those geometrical objects, which we want to detect; • P (I|θ) is the posterior probability: the distribution element of objects identiers under the assumption of already detected objects; • P (θ)is the prior probability: it is the formalization of our intuition about the possibility of the detection of an object.
In our case the value of this probability is dened statistically after the quantitative data analysis, after the supervision of such parts in orthodox churches.
All conditional probabilities are learned from training datasets.
The expression P (I|θ) is called likelihood; it is a probability of a known data supervision by the xation of certain identiers.Thus, our task is to nd the maximum a posterior hypothesis θ, by which arg max θ P (θ|I).
If we set P (I|θ) of likelihood functions according to equation ( 1), the decision problem will be solved correctly.However, the establishment process of these functions is not a trivial procedure.These distributions can be rather difcult.
If we add some additional restrictions, the problem of P (I|θ)function calculation can be simplied.By the conditional independence a1, a2, . .., an of the objects identiers, which dene I, we have: When a1 and a2 influence on θ at the same time (fig.2c) it is possible to express it as P (a1, a2, θ) = P (a1)P (a2)P (θ|a1, a2) In this case we receive a likelihood function which depends on the intended value, that, obviously, does not yield the correct decision.In this case the likelihood-function will be directly calculated directly.
A similar approach will allow us to optimize the procedure of method choice for object extrapolation.

Processing principles
Let us consider the graph which describes an input church as a special type of a cellular automaton.This graph is a lattice of cellular automaton, in which vertices are the automata cells.The cellular automaton can be defined as a set of final automatas with concrete state in discrete time t: A detection of component geometries changes the automaton states of each cell in the neighborhood according to the transition rule: N is the set of automatas that constitute a neighborhood.At time t we have 3 kinds of cells: 1. cells with unambiguously defined state (e.g.detected geometry); 2. "pending" cells which incident (but do not belong) to defined cells; 3. "empty" cells which do not incident and do not belong to defined cells.
In our case, we set an initial state randomly.It is reasonable to set the most probable state as initial state: it means, the search is statistical driven starting with the most likely geometry for each component.
The transition function will be defined: if the vertex B does not belong to N , there are edges from the vertex B incidental to the vertices from N ("pending cells"): The state of the vertex is If the vertex B belongs to N , the state of a vertex remains the same with the probability of 1 (almost sure event).
If the vertex B has no edge with any vertex from N , the state of a vertex remains the same with the probability of 1.
Applying the Maximum-Likelihood method, we choose the state in the pending vertices respectively to the maximum of the corresponding probabilities B k = {Bi, max(pi) = p k } and check for the correctness.It denes appropriate processes in all pending vertices, in which we detect the geometry of church structural element by the method chosen according to a state.The following results are possible: 1.The geometry in the pending vertex has been detected.In this case we attach a vertex to the N-set.
2. The geometry in the pending vertex has not been detected with the method chosen according to a state (we define this state as B1 without loss of generality).
Let us transform the transition function: If the vertex B does not belong to N , there are edges from the vertex B incidental to vertices from N ("pending vertices") and on the previous step this vertex was not a pending vertex: The state of the vertex is If the vertex B does not belong to N , there are edges from the vertex B incidental to the vertices from N ("pending vertices") and on the previous step this vertex was a pending vertex: The state of the vertex is If the vertex B belongs to N , the state of a vertex remains the same with the probability of 1.
If the vertex B has no edge with any vertex from N , the state of a vertex remains the same with the probability of 1.
This iterative process yields to a distinct solution since all cells will nally change its state to "unambiguously dened".In our case it is convenient to visualize the state of a vertex with a color.To each state of Bk a specic color is assigned.Thus, the presented probabilistic automaton presents the process of vertex coloring in the graph.

EXAMPLE
Let us consider the example of algorithm realization.In this section we will present the workow of the version of our probabilistic automaton.At the moment our probabilistic automaton is singlestep.Unlike classical probabilistic automatas we make actions on the correctness control of the automaton state changes between processing steps.We check, whether the offered extrapolation method is correct (in other words, whether the vertex is painted in the correct color on a processing step).The task of the automaton is to paint all vertices in a correct color.We will dene transition functions according to the Bayesian network rules.
It will be taken into account, that the topology and geometry of some vertices is already known.The vertices Kr, HK, T r, St, D, X, OAr, Dr are painted in the color that corresponds to their certain geometry.The vertices HA, D, OA l ,D l have not been detected yet.Therefore the lack of color (or the lack of any knowledge about the geometry of this object) is represented with the gray color of the vertices (cf.Fig. 3).
The operation principle of the cellular automaton in our case is as follows: we start the parallel operation of nal automaton for each gray vertex.But only for two vertices the automatic functions, which transforms a vertex color, will be uncommon.In our case we have two such gray tops: HA and OA l .There are no incidental edges for them, which connect gray and colored vertices.
Other gray vertices actions of automatic functions will lead to the coloring of those vertices in the same gray color.
Let us consider the case when we need to dene the HA-vertex geometry.We will describe the operation of the corresponding automaton by identifying the color of this vertex.There are three approaches or three ways of extrapolation for its identication.It means that it is necessary to dene which of the three colors -e.g.red, blue or green -will correspond to the real existing geometry.
In our algorithm we will apply the Bayesian rules.
Let us define the following variations: θ1 -is the event (or incident), which determines the red color of HA -vertex, the color of θ2 -blue or the color of θ3 -green.The choice of i-vertex will be conditioned by the maximum of the expression P (θ|I), i = 1, 2, 3.The I -event means in our case that incidental to HA-vertex detected vertices X and OAr are painted in red color.Probabilities P (θ1), P (θ2), P (θ3), P (I) can be found using the statistical data presented in Table 1.Then it is necessary to find the likelihood functions P (I|θi), i = 1, 2, 3 using the properties of Bayesian networks, which we have described in section 2. It is important to notice that P (I|θ) is equal to P (X|θ) × P (OAr|θ) in our case.

Vertex
The maximum of likelihood function P (θ|I) is reached at θ = θ1.Therefore the vertex HA has to be painted in red color.Thus, we have defined the transition function for a new state.Further the following cases are possible.
1.The color has defined a correct object extrapolation and we have received a geometry as the red color of HA-vertex by the detection process.The quantity of vertices (not detected objects) has been decreased and the next cycle of detection process have to been started.
2. It was a mistake and the offered extrapolation method was incorrect.It leads to the recalculation of Tab. 1 (the amount of sequences decreases because we do not consider a wrong steps), values in the Table 2 will be recalculated conditioned by the refusal of a line with an incorrect color.The further searching procedure of the P (θ|I) -maximum proceeds.
The quantity of lines in the Table 2 is final.Therefore, this process is final by the cycle iteration and will lead to the final detection of the object.
The second nal parallel started automaton will dene the color of OAl-vertex.Let us state that a1 is an event that OA l -vertex is "red", a2 -OA l -vertex is "blue", a3 -"green".As there is only one vertex -X, painted in red color is incidental to OA l vertex, we will consider three conditional probabilities P (OA l = a1|X = red) = P (a1, X) P (X) , P (OA l = a2|X = red) = P (a2, X) P (X) , P (OA l = a3|X = red) = P (a3, X) P (X) , P (X = red) means the selection of those churches, which are described in our knowledge base, with the attribute X = red.In our case of P (X = red) = 0, 46.
P (OA l = ai, X = red) with i = 1, 2, 3 is the selection of those churches from the knowledge base, which have OA l = ai and X = red at the same time.
From P (OA l = a1, X = red) = 0, 413, P (OA l = a2, X = red) = 0, 282, and P (OA l = a3, X = red) = 0, 305 follows P (OAl = a1|X = red) = max Therefore, we identify the color of OA l as red, and further, we apply the corresponding way of extrapolation.If the way corresponding to red color is confirmed, the vertex is recolored in a red color, if not, then in the following step the number of alternatives decreases by one and the process repeats.The process is final (Fig. 4).

CONCLUSION
The presented method allows an automatic reconstruction of the whole complex object with missed components on the basis of iterative detected geometries of its components.The method principles are universal and applicable for other types of objects (it needs other library of structural components and component relation in graph).We developed the mathematical theory and showed the feasibility of the approach for simulated data of stone Russianorthodox churches.Further work will concentrate in two directions.On the one hand, the empirical work on the analysis of the historical development of the churches will be continued.On the other hand, the cellular automaton will be realized for larger and more complex building models.
Figure 2: Subgraphs of the probability connections.pointers on the graph edges are suspended in most of the cases of this article.The graph of Fig. 2a corresponds to the expression in eq.(2): P (a1, θ, a2) = P (a1)P (θ|a1)P (a2|θ).

Table 1 :
Simulated statistical data: each line of the table contains the number of examples to generate the conditional probabilities.

Table 2 :
Deduction of the Likelihood function.