A Modified Algorithm for the Computation of the Covariance Matrix Implied by a Structural Recursive Model with Latent Variables Using the Finite Iterative Method

Structural Equation Modeling (SEM) is a statistical technique that assesses a hypothesized causal model by showing whether or not, it fits the available data. One of the major steps in SEM is the computation of the covariance matrix implied by the specified model. This matrix is crucial in estimating the parameters, testing the validity of the model and, make useful interpretations. In the present paper, two methods used for this purpose are presented: the Jöreskog’s formula and the finite iterative method. These methods are characterized by the manner of the computation and based on some a priori assumptions. To make the computation more simplistic and the assumptions less restrictive, a new algorithm for the computation of the implied covariance matrix is introduced. It consists of a modification of the finite iterative method. An illustrative example of the proposed method is presented. Furthermore, theoretical and numerical comparisons between the exposed methods with the proposed algorithm are discussed and illustrated.


Introduction
Structural Equation Modeling (SEM) is a set of statistical techniques to test the conformity of a hypothesized causal model to some available data [1,2,4]. Structural equation models are characterized by the presence of both observed and latent variables. As a consequence, their procedures are more complicated compared to the classical statistical methods such as linear regression, ANOVA,... In fact, most of the classical models lie in the general case of SEM [2]. Currently, SEM is widely used in many areas. To name a few: environmental sustainability [26], ecological studies [8,17], social sciences [9,24] and psychology [3]. SEM combines Path Analysis [18,19,20,21,9] with Confirmatory Factor Analysis [25,16] by taking into account that some variables are not directly observed. The general idea of SEM is that the covariance matrix of the observed variables can be expressed as a function of the model parameters [10,12,11,14,13,15]. In other words: If Σ is the population covariance matrix of the model variables, and Σ(θ) is the covariance matrix implied by the model. Then the fundamental hypothesis of SEM can be expressed as: (4) Such that: ζ is the vector of disturbances that refer to all non-included causes in the structural model and δ and ϵ are the vectors of the measurement errors of each measurement model. All those vectors are error terms.
The following matrix notations are adopted: B is the matrix (p × p) of structural coefficients between endogenous variables. Γ is the matrix (p × q) of structural coefficients between exogenous and endogenous variables. Λ x is the matrix (n x × q) of loadings between exogenous latent variables and thier associated manifest variables. Λ y is the matrix (n y × p) of loadings between endogenous latent variables and thier associated manifest variables. The covariance matrix of the vector of exogenous variables is denoted Φ (q × q). The covariance matrix of the vector of disturbances (i.e. ζ) is denoted Ψ (p × p). The covariance matrix of the vector of measurement errors associated with exogenous variables (i.e δ) is denoted Θ δ (n x × n x ). Finally, The covariance matrix of the vector of measurement errors associated with endogenous variables (i.e ϵ) is denoted Θ ϵ (n y × n y ).
Another important aspect of SEM is the path diagram. It is defined as the graphical representation of the model. Many researchers [2] think that it can be more representative then the structural equations. A path diagram is a directed graph (example Figure 1). The vertices correspond to variables (latent variables, observed variables, and error terms). It is an illustration of the direction of a hypothesized causation between each pair of variables. The edges are i) the one-headed arrow which means the sense of causality between variables and; ii) the two-headed arrow which means that the relationship between its ending vertices (variables) is not analyzed by the model (unanalyzed association). The relations described by the path diagram is complete so that, for example, the nonexistence of an arrow between two variables means the nonexistence of a direct association between them. An SEM model is said to be recursive if for every pair of variables if the one causes the second then the second can never be a cause of the first. Otherwise, the model is said to nonrecursive. mathematically, a model is said to be recursive if there exists a permutation of the lines of the matrix B so that it will be strictly lower triangular (in all what follows, we will consider that the matrix B permuted so that it is strictly lower triangular for a recursive model). Graphically, a model is said to be recursive if its path diagram acyclic (i.e contain no feedback loops nor reciprocal causations). SEM models consist of 6 major steps: model specification, model identification, model estimation, model validation, model modification and, interpretation [27]. The estimation is a crucial step in SEM. It consists of expressing the Covariance matrix implied by the model (i.e. Σ) as a function of model (the vector of model parameters is denoted θ) parameters and then find the estimates of those parameters. This is usually done by minimizing a discrepancy function between the population covariance The main topic of the present paper is: For an SEM model, given its parameters, how to compute Σ? The first answer to this question was proposed by [10]. Then [5] and [6] proposed an iterative method called Finite Iterative Method (FIM) for Path Analysis models (SEM with observed variables). FIM has been then extended to SEM models [7]. This paper aims to introduce a modified FIM algorithm which has some advantages, that will be discussed later, compared to the previous methods. For this purpose, the manuscript is partitioned as follows: section 2 discusses briefly the implied covariance matrix and the previous methods used for its computation. section 3 exposes the modified algorithm of the FIM method with its demonstration. sections 4 and 5 illustrate the seen methods as well as the comparison between them through an example. Finally, section 6 concludes with discussions and perspectives.

The implied covariance matrix: the Jöreskog's formula and the finite iterative method
The present section introduces the concept of the implied Covariance matrix and the methods used for its computation, namely Jöresko's formula [10] and the Finite Iterative Method [7].

Definition
The estimation step consists of finding the parameters that make the model fit the data. In other words, it aims to find values of the parameters that make the structure of the model as close as possible to the data structure. The model structure is characterized by the assumption that the distribution of the observed variables is multivariate † Another approach in SEM PLS Path Modeling. This approach introduced by [28] is the variance-based method. While the treated approach in this manuscript is the covariance-based method introduced by [10]. The authors suggest [33] to the reader for PLS Path Modeling approach 362 A MODIFIED ALGORITHM FOR THE COMPUTATION OF THE IMPLIED COVARIANCE MATRIX and sufficiently well described by its mean and covariance [11]. This is translated by the fundamental hypothesis of SEM (1). The implied covariance matrix is the theoretical covariance matrix between variables generated by an SEM model. It is written as a function of the free model parameters. For instance, the covariance matrix implied by the model defined by (2), (3) and, (4) is written as follows: On the one hand, it can be determined algebraically by writing the vector of endogenous variables as a function of the vector of exogenous ones [10]. On the other hand, it can also be constructed graphically using the tracing rules of Path Analysis [22,29], by calculating every covariance (or correlation in Path Analysis models) between each pair of model variables. [5,6], and [7] proposed the Finite Iterative Method, which uses these rules to build an implied correlation matrix with an iterative algorithm. Further discussion in termes of the comparison between Jöreskog's formula and the Finite iterative method is given in [23].

Jöreskog's formula
As above mentioned, The Jöreskog's formula is a compact representation of the Implied covariance matrix given in (5). In order to compute the implied covariance matrix, Jöreskog starts from the following assmptions: • All variables are supoosed to be centred to their means. • All disturbances and measurment errors are uncorrelated with each other. That means : • Exogenous variables are uncorrelated with disturbances and with measurment errors. That is : The equation (2) can be written in the compact form: Using equation (3), (4) and (6) and taking the mathematical expectation of XX t , XY t and Y Y t , the implied covariance matrix in (5) will be :

The finite iterative method
The finite Iterative Method is an iterative method which does not take into account the error terms in the computation of the implied correlation matrix. To be more clear, the parameters which are related to the error terms are constrained (set as a function of the rest of model variables). The starting point of the method is to separate the model into two types of submodels: a) the structural model (the inner model) and b) the measurement models. (see Figure 2 as an example). In addition to that FIM has its own notations and assumptions. Indeed, All latent variables are represented by ξ (i.e. ξ = (ξ 1 , . . . , ξ q , η 1 , . . . , η p )). The vector of all manifest variables is denoted X and the vector all measurement errors is denoted δ. Finally for a matrix M , the following notation is adopted such that: is the matrix of the structural coefficients. Besides, the equations (3) and (4) become: is the matrix ((n x + n y ) × (n x + n y )) of all the loadings. The FIM algorithm proposed by [7] is based on the following assumptions: • The model is supposed to be recursive. • All manifest variables are centred and all latent variables are standardized.
• Disturbances are uncorrelated with the latent variables explaining its associated endogenous latent variables. In other words: The first stage of the computation of the covariance matrix implied by a structural recursive model is the implied correlation matrix R (LV ) of the structural model (since the latent variables are standardized) using Algorithm 1 proposed by [5].
Then the implied covariance matrix of the whole model is given as follows: such that R LV = E[ξξ t ] is the matrix computed in the first stage by Algorithm 1. Besides, elements of Θ are constrained by the following equation:  [7] initialize: R (LV ) 1:q,1:q = Φ and j = 1

A modified Algorithm of FIM
In this section we will present another view to the extension of FIM to SEM models which have more simplistic algorithm and less restrictive assumptions compared the FIM. The main Idea is to adapt Algorithm 1 with new input so it can be applied in one stage instead of two stages as in section (2). The algorithm (Algorithm 2 bellow) will computate the implied correlation matrix of all model variables and extract the covariance matrix of the observed variables.

Notations and assumptions
First, all variables, latent and observed, that appear dependent in the equations (2), (3) and (4) (i.e. η, X and Y ) will be gatherd in a vector denoted η ⋆ . The same for disturbances and measurement errors (i.e. ζ, δ, ϵ), they will be gathered in one vector of errors terms denoted ζ ⋆ . Two new matrices B ⋆ : ((p + n x + n y ) × (p + n x + n y )) and Γ ⋆ : ((p + n x + n y ) × q) will be defined as follows: Indeed, the equations (2), (3) and (4) will be gathered in the follwing one: Which can also written as : Where A ⋆ , a matrix of order ((p + n x + n y ) × (q + p + n x + n y )) defined as : The necessary assumptions for our method are: • The model is supposed to be recursive. • All manifest variables are centred to their mean.
• In each equation, the error term is uncorrelated with the independent variables. In other words, using (15)

A modified FIM algorithm
Our proposed algorithm (Algorithm 2) is based on the implied covariance matrix of the whole model variables (latent and manifest except error terms). It is expressed as follws: Once the matrix (17) is obtained, we extract from it the covariance matrix of the observed variables ( Σ). The model became defined by (14).

Theorem 1
For a given recursive structural model, if Σ and Σ M F IM are respectivelly the implied covariance matrix and the matrix computed using Algorithm 2 . Then we have the following equality:

Proof
Consider a recursive structural model defined by equations (2), (3) and (4). Using notations in 3.1, the model became deined by (14). Now let Σ ⋆ be its implied covariance matrix and Σ M F IM ⋆ be the matrix computed using algorithm 2. The idea is to prove, first of all, that: Since: 1. The matrix Σ M F IM ⋆ is symetric by its construction from Algorithm 2, 2. the matrix Σ ⋆ is symetric by the definition of a covariance matrix, 3. the diagonal of the matrice are equal: Then, proving (22) is reduced to proving : • From the definition of the covariance matrix of model variables (17) : There are two cases for j + 1:

In both cases :
A ⋆ j+1,q+p+nx+ny = 0 using this equation in (26) we get : Now we put (27) in (25), we use the assumption that ζ ⋆ j+1 is uncorrelated with the independent variables in the j th equation and we get : From (28) and (24), we conclude that (23) is correct, which means :(22) is correct. By the principle of induction,from (20), (21) and, (22) : (19) is correct. Finally, we apply the fifth step of Algorithm 2 by multiplying both sides of (19) with G at the left and G t at the right: Therefore: from (29), (18) is correct.
In order to simplify Algorithm 2, we propose the following corollary to make it expressed as a recursive function:

Corollary 1
Let Σ be the covariance matrix implied by a structural recursive model, and let be a recursive function such that:

Example for illustration
In this section will illustrae this proposed method using Figure 3 for a better understanding of our proposed method (Algorithm 2). The model contain, for representability and simplicity reasons, tow latent variables (exogenous variable ξ 1 and endogenous variable η 1 ) and for manifest variables (tow variables related to the exogenous variable X 11 and X 12 and two variables related to the endogenous variable Y 11 and Y 12 ) notations of section 3.1 are replaced, for the same reasons, by the following ones: 11 and, e = λ (y) 12 Hence the matrix A ⋆ is given as follows: Denote the variance of ξ 1 by ϕ 1 and the variance of the variable η ⋆ by σ 2 η ⋆ for each η ⋆ in { η 1 , X 11 , X 12 , Y 11 , Y 12 }. Applying steps 0 to 4 of Algorithm 2 lead to the folowing matrix: Applying the fifth step of Algorithm 2 lead to the implied covariance matrix as follows :

Theoritical and numerical comparison
The present section exposes some comparisons between the methods discussed previously. More precisely, we will show some theoretical comparison in terms of the manner of the computation as well as the assumptions adopted by each method. Then we will show, through a simulation example the practical differences between them in different situations.

Theoritical comparison
The differences between Jöreskog's formula, the Finite Iterative Method, and our proposed modification of FIM can be seen in two main levels: • Compuatational level: At this level, Jöreskog's formula compute the covariance matrix implied by a structural model by block. It gives a compact expression of our desired covariance matrix. In addition, it requires the inversion of the matrix (I − B) [7]. On the other side, both algorithms (Algorithm 1 and Algorithm 2) construct the covariance matrix iterativlly. Jöreskog's formula requires Γ, B, Λ x , Λ y , Φ, Ψ, Θ δ and, Θ ϵ . For its part, FIM requires Γ, B, Λ x , Λ y , Φ, Θ δ and, Θ ϵ . However, using the consraint in (11) will make Θ δ and Θ ϵ not needed for the compuatation [7]. Our proposed method have the same model parameters as the FIM plus the variances of the manifest and latent variables. In fact, in FIM these parameters are not took into account because they are constrained to be equal to the empirical variances for manifest variables and 1 for latent variables. For us, no need to impose this constraint nor the constraint (11) explicitly.
• Assumptions level: The main advantage of Jöreskog's formula is that the model does not need to be recursive. A less restrictive assumption adopted by Jöreskog is that the matrix (I − B) is not singular. On the other hand, FIM and our proposed method require a recursive model. However, research on the computation of the covariance matrix implied by a structural nonrecursive model is underway. Besides, Jöreskog assumes that all error terms (i.e. measurement errors and disturbances) are uncorrelated with each other and uncorrelated with all exogenous variables. FIM assumes that measurement errors are uncorrelated with each other and each disturbance is uncorrelated with all independent variables that are in the same equation containing this error term. This is less restrictive compared to Jöreskog's formula. Least restrictive assumptions are ours. In fact, we assume each error term is uncorrelated with all independent variables that are in the same equation containing this error term without adding the assumption that measurement errors are uncorrelated with each other. It should be mentioned that if the causal model is true at the population level, all the covariances computed by different methods should be equal. This is because the covariance matrix is unique for each model. These differences seen between the calculated covariance matrices are due to: a) the assumptions in which based each method and b) the constraints (which are correct in the true model at the population level) that are used in each computation. The second constraint is in terms of the variances of disturbances, we explain this constraint as follows: From (8), ∀j ∈ {1, . . . , p} : Therefore : σ 2 ηj = E[η 2 j ] = A 1,1:q+j−1 Σ 1:q+j−1,1:q+j−1 A t 1,1:q+j−1 + ψ j Which means : In addition to that, for identification reasons, one may set free the variances of the latent variables and scale it by making one of its loadings constrained to one [15,2]. This is possible when applying Jöreskog's formula or Algorithm 2. However, since the variances of latent variables are set to one in the finite iterative method (Algorithm 1), we can not constrain the loadings to one.

Numerical comparison
In this section, we will compare all methods using a simulated example. We have generated, in three stages, 100 vectors of model parameters ( the vector of model parameters θ contain all free parameters of the matrices B, Γ, Λ x , Λ y , Ψ, Θ δ and, Θ ϵ ). We calculate the implied covariance matrix via i) Jöreskog's formula, ii) the finite iterative method and, iii) our proposed method. In the first stage, we generated the parameters such that the measurement errors are constrained by the equation (11). In the second stage, the disturbances are constrained by  the formula (34). While the third simulation will make the variances constrained to one ( the second assumption in FIM method)). The adapted model is Figure 3. After that, we compute the distance between these matrices using the discrepancy function defined for given two squared matrices M 1 and M 2 : Figure 4 shows the results of the simulation. Practically, the methods lead to closed matrices (the maximum difference obtain in 300 simulations is .0078). When the error terms (measurement errors and disturbances) are constrained the distance between the method became closer. This is because we impose the constraints to be valid (as in the true model at the population level as aforementioned). When the variances are constrained to one (the second assumption of FIM is valid), Our proposed method became practically identical to FIM (the maximum distance between them is bellow 2 × 10 −32 ). In this third simulation, the curve of the difference between matrices computed by the finite iterative method and our proposed method (acronym FIM-MFIM) does not show because it's near zero. Besides, the curve of the difference between the matrice computed by Jöreskog's formula and our proposed method (acronym Jöreskog-MFIM) does not show because it's approximately identical to the difference between Jöreskog's formula and FIM (acronym Jöreskog-FIM).

Summary, discussion and perspectives
We presented SEM models, their fields of application, their notation, the fundamental hypothesis of SEM and, the typology of variables and models. We discussed the steps of SEM and focused on the estimation part. Especially the computation of the covariance matrix implied by an SEM model. Two major methods are exposed: the Jöreskog's formula [10] and the Finite Iterative Method [7]. We showed that Jöreskog' formula computes the implied covariance matrix by block, it requires the matrix (I − B) to be invertible instead of the recursiveness of the model. It assumes no correlation among all error terms and between exogenous variables and error terms. No need for the latent variables to be standardized.
We presented the Finite Iterative Method for SEM models. It is an extension of the work of [5,6]. The method computes the implied covariance matrix through two steps: the first step is the computation of the implied correlation matrix of the structural model using Algorithm 1. Then, the second step is the computation of the covariance matrix of the model using (10). It assumes: all latent variables are standardized, noncorrelation among measurement errors and, each disturbance is uncorrelated with all independent variables containing this disturbance.
We proposed a modified algorithm (Algorithm 1) for the computation of the covariance matrix using FIM. The advantages of this algorithm are: i) it computes the desired matrix in one step without separating the model in the structural part and the measurement part and ii) it is less restrictive in terms of the assumptions compared to both methods. The only required assumptions are the regressional assumptions (in each equation, the error term is uncorrelated with the independent variables in that equation).
A Python program is written to implement all three methods as functions. In the case of our proposed method, we implemented it using the function f in Corollary 1. The program is available upon request.
In terms of perspectives, current research on the development of FIM for nonrecursive models is on. This work will focus on finding some characteristics of a nonrecursive model that make it possible to compute its implied covariance matrix. In addition to that, we think that FIM will serve to compute the implied covariance matrix for formative models (this idea is already described by [30]). This idea is under study and will introduce new researchers to adapt the covariance-based approach in their studies. Especially in the fields that use PLS Path Modeling (variance-based approach of SEM) due to its advantage in these kinds of models. The authors suggest, for instance, [32] and [31] where the differences between reflective and formative approaches are explained. The more recent advancement on this topic will be considered by the authors for the future development of formative blocks in FIM.