Consider fitting a linear equation to two observed variables , Y and X . Simple linear regression uses the model of a particular form, labeled for purposes of discussion, as Model Form A.
where ± and ² are coefficients to be estimated and E Y is an error term . If the values of X are fixed, the values of E Y are assumed to be independent and identically distributed realizations of a normally distributed random variable with mean zero and variance Var( E Y ). If X is a random variable, X and E Y are assumed to have abivariate normal distribution with zero correlation and variances Var( X ) and Var( E Y ), respectively. Under either set of assumptions, the usual formulas hold for the estimates of the coefficients and their standard errors (see Chapter 2, Introduction to Regression Procedures, ).
In the REG or SYSLIN procedure, you would fit a simple linear regression model with a MODEL statement listing only the names of the manifest variables:
proc reg; model y=x; run;
You can also fit this model with PROC CALIS, but you must explicitly specify the names of the parameters and the error terms (except for the intercept, which is assumed to be present in each equation). The linear equation is given in the LINEQS statement, and the error variance is specified in the STD statement.
proc calis cov; lineqs y=betax+ex; std ex=vex; run;
The parameters are the regression coefficient beta and the variance vex of the error term ex . You do not need to type an * between beta and x to indicate the multiplication of the variable by the coefficient.
The LINEQS statement uses the convention that the names of error terms begin with the letter E, disturbances (errors terms for latent variables) in equations begin with D, and other latent variables begin with F for factor. Names of variables in the input SAS data set can, of course, begin with any letter.
If you leave out the name of a coefficient, the value of the coefficient is assumed to be 1. If you leave out the name of a variance, the variance is assumed to be 0. So if you tried to write the model the same way you would in PROC REG, for example,
proc calis cov; lineqs y=x;
you would be fitting a model that says Y is equal to X plus an intercept, with no error.
The COV option is used because PROC CALIS, like PROC FACTOR, analyzes the correlation matrix by default, yielding standardized regression coefficients. The COV option causes the covariance matrix to be analyzed , producing raw regression coefficients. See Chapter 2, Introduction to Regression Procedures, for a discussion of the interpretation of raw and standardized regression coefficients.
Since the analysis of covariance structures is based on modeling the covariance matrix and the covariance matrix contains no information about means, PROC CALIS neglects the intercept parameter by default. To estimate the intercept, change the COV option to UCOV, which analyzes the uncorrected covariance matrix, and use the AUGMENT option, which adds a row and column for the intercept, called INTERCEPT, to the matrix being analyzed. The model can then be specified as
proc calis ucov augment; lineqs y=alpha intercept + betax+ex; std ex=vex; run;
In the LINEQS statement, intercept represents a variable with a constant value of 1; hence, the coefficient alpha is the intercept parameter.
Other commonly used options in the PROC CALIS statement include
MODIFICATION to display model modification indices
RESIDUAL to display residual correlations or covariances
STDERR to display approximate standard errors
TOTEFF to display total effects
For ordinary unconstrained regression models, there is no reason to use PROC CALIS instead of PROC REG. But suppose that the observed variables Y and X are contaminated by error, and you want to estimate the linear relationship between their true, error-free scores. The model can be written in several forms. A model of Form B is as follows .
This model has two error terms, E Y and E X , as well as another latent variable F X representing the true value corresponding to the manifest variable X . The true value corresponding to Y does not appear explicitly in this form of the model.
The assumption in Model Form B is that the error terms and the latent variable F X are jointly uncorrelated is of critical importance. This assumption must be justified on substantive grounds such as the physical properties of the measurement process. If this assumption is violated, the estimators may be severely biased and inconsistent.
You can express Model Form B in PROC CALIS as follows:
proc calis cov; lineqs y=beta fx + ey, x=fx + ex; std fx=vfx, ey=vey, ex=vex; run;
You must specify a variance for each of the latent variables in this model using the STD statement. You can specify either a name, in which case the variance is considered a parameter to be estimated, or a number, in which case the variance is constrained to equal that numeric value. In general, you must specify a variance for each latent exogenous variable in the model, including error and disturbance terms. The variance of a manifest exogenous variable is set equal to its sample variance by default. The variances of endogenous variables are predicted from the model and are not parameters. Covariances involving latent exogenous variables are assumed to be zero by default. Covariances between manifest exogenous variables are set equal to the sample covariances by default.
Fuller (1987, pp. 18 “19) analyzes a data set from Voss (1969) involving corn yields ( Y ) and available soil nitrogen ( X ) for which there is a prior estimate of the measurement error for soil nitrogen Var ( E X ) of 57. You can fit Model Form B with this constraint using the following SAS statements.
data corn(type=cov); input _type_ $ _name_ $ y x; datalines; n . 11 11 mean . 97.4545 70.6364 cov y 87.6727 . cov x 104.8818 304.8545 ; proc calis data=corn cov stderr; lineqs y=beta fx + ey, x=fx + ex; std ex=57, fx=vfx, ey=vey; run;
In the STD statement, the variance of ex is given as the constant value 57. PROC CALIS produces the following estimates.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation y = 0.4232*fx + 1.0000 ey Std Err 0.1658 beta t Value 2.5520 x = 1.0000 fx + 1.0000 ex Variances of Exogenous Variables Standard Variable Parameter Estimate Error t Value fx vfx 247.85450 136.33508 1.82 ey vey 43.29105 23.92488 1.81 ex 57.00000
PROC CALIS also displays information about the initial estimates that can be useful if there are optimization problems. If there are no optimization problems, the initial estimates are usually not of interest; they are not reproduced in the examples in this chapter.
You can write an equivalent model (labeled here as Model Form C) using a latent variable F Y to represent the true value corresponding to Y .
The first two of the three equations express the observed variables in terms of a true score plus error; these equations are called the measurement model. The third equation, expressing the relationship between the latent true-score variables, is called the structural or causal model. The decomposition of a model into a measurement model and a structural model (Keesling 1972; Wiley 1973; J reskog 1973) has been popularized by the program LISREL (J reskog and S rbom 1988). The statements for fitting this model are
proc calis cov; lineqs y=fy + ey, x=fx + ex, fy=beta fx; std fx=vfx, ey=vey, ex=vex; run;
You do not need to include the variance of F Y in the STD statement because the variance of F Y is determined by the structural model in terms of the variance of F X , that is, Var( F Y )= ² 2 Var( F X ).
Correlations involving endogenous variables are derived from the model. For example, the structural equation in Model Form C implies that F Y and F X are correlated unless ² is zero. In all of the models discussed so far, the latent exogenous variables are assumed to be jointly uncorrelated. For example, in Model Form C, E Y , E X ,and F X are assumed to be uncorrelated. If you want to specify a model in which E Y and E X , say, are correlated, you can use the COV statement to specify the numeric value of the covariance Cov( E Y , E X ) between E Y and E X , or you can specify a name to make the covariance a parameter to be estimated. For example,
proc calis cov; lineqs y=fy + ey, x=fx + ex, fy=beta fx; std fy=vfy, fx=vfx, ey=vey, ex=vex; cov ey ex=ceyex; run;
This COV statement specifies that the covariance between ey and ex is a parameter named ceyex . All covariances that are not listed in the COV statement and that are not determined by the model are assumed to be zero. If the model contained two or more manifest exogenous variables, their covariances would be set to the observed sample values by default.
Unfortunately, if you try to fit models of Form B or Form C without additional constraints, you cannot obtain unique estimates of the parameters. These models have four parameters (one coefficient and three variances). The covariance matrix of the observed variables Y and X has only three elements that are free to vary, since Cov( Y , X )=Cov( X , Y ). The covariance structure can, therefore, be expressed as three equations in four unknown parameters. Since there are fewer equations than unknowns, there are many different sets of values for the parameters that provide a solution for the equations. Such a model is said to be underidentified.
If the number of parameters equals the number of free elements in the covariance matrix, then there may exist a unique set of parameter estimates that exactly reproduce the observed covariance matrix. In this case, the model is said to be just identified or saturated .
If the number of parameters is less than the number of free elements in the covariance matrix, there may exist no set of parameter estimates that reproduces the observed covariance matrix. In this case, the model is said to be overidentified. Various statistical criteria, such as maximum likelihood, can be used to choose parameter estimates that approximately reproduce the observed covariance matrix. If you use ML, GLS, or WLS estimation, PROC CALIS can perform a statistical test of the goodness of fitof the model under the certain statistical assumptions.
If the model is just identified or overidentified, it is said to be identified. If you use ML, GLS, or WLS estimation for an identified model, PROC CALIS can compute approximate standard errors for the parameter estimates. For underidentified models, PROC CALIS obtains approximate standard errors by imposing additional constraints resulting from the use of a generalized inverse of the Hessian matrix.
You cannot guarantee that a model is identified simply by counting the parameters. For example, for any latent variable, you must specify a numeric value for the variance, or for some covariance involving the variable, or for a coefficient of the variable in at least one equation. Otherwise, the scale of the latent variable is indeterminate, and the model will be underidentified regardless of the number of parameters and the size of the covariance matrix. As another example, an exploratory factor analysis with two or more common factors is always underidentified because you can rotate the common factors without affecting the fit of the model.
PROC CALIS can usually detect an underidentified model by computing the approximate covariance matrix of the parameter estimates and checking whether any estimate is linearly related to other estimates (Bollen 1989, pp. 248 “250), in which case PROC CALIS displays equations showing the linear relationships among the estimates. Another way to obtain empirical evidence regarding the identificationofa model is to run the analysis several times with different initial estimates to see if the same final estimates are obtained.
Bollen (1989) provides detailed discussions of conditions for identificationinavarietyofmodels.
The following example is inspired by Fuller (1987, pp. 40 “41). The hypothetical data are counts of two types of cells, cells forming rosettes and nucleated cells , in spleen samples. It is reasonable to assume that counts have a Poisson distribution; hence, the square roots of the counts should have a constant error variance of 0.25.
You can use PROC CALIS to fit a model of Form C to the square roots of the counts without constraints on the parameters, as displayed in following statements. The option OMETHOD=QUANEW is used in the PROC CALIS statement because in this case it produces more rapid convergence than the default optimization method.
data spleen; input rosette nucleate; sqrtrose=sqrt(rosette); sqrtnucl=sqrt(nucleate); datalines; 4 62 5 87 5 117 6 142 8 212 9 120 12 254 13 179 15 125 19 182 28 301 51 357 ;
proc calis data=spleen cov omethod=quanew; lineqs sqrtrose=factrose + err_rose, sqrtnucl=factnucl + err_nucl, factrose=beta factnucl; std err_rose=v_rose, err_nucl=v_nucl, factnucl=v_factnu; run;
This model is underidentified. PROC CALIS displays the following warning:
WARNING: Problem not identified: More parameters to estimate(4) than given values in data matrix(3).
and diagnoses the indeterminacy as follows:
NOTE: Hessian matrix is not full rank. Not all parameters are identified. Some parameter estimates are linearly related to other parameter estimates as shown in the following equations:
v_nucl = -10.554977 - 0.036438 * beta + 1.00000 * v_factnu + 0.149564 * v_rose
The constraint that the error variances equal 0.25 can be imposed by modifying the STD statement:
proc calis data=spleen cov stderr; lineqs sqrtrose=factrose + err_rose, sqrtnucl=factnucl + err_nucl, factrose=beta factnucl; std err_rose=.25, err_nucl=.25, factnucl=v_factnu; run;
The resulting parameter estimates are displayed in Figure 13.2.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation factrose = 0.4034*factnucl Std Err 0.0508 beta t Value 7.9439 Variances of Exogenous Variables Standard Variable Parameter Estimate Error t Value factnu cl v_factnu 10.45846 4.56608 2.29 err_rose 0.25000 err_nucl 0.25000
This model is overidentified and the chi-square goodness-of-fit test yields a p -value of 0.0219, as displayed in Figure 13.3.
Covariance Structure Analysis: Maximum Likelihood Estimation Fit Function 0.4775 Goodness of Fit Index (GFI) 0.7274 GFI Adjusted for Degrees of Freedom (AGFI) 0.1821 Root Mean Square Residual (RMR) 0.1785 Parsimonious GFI (Mulaik, 1989) 0.7274 Chi-Square 5.2522 Chi-Square DF 1 Pr > Chi-Square 0.0219 Independence Model Chi-Square 13.273 Independence Model Chi-Square DF 1 RMSEA Estimate 0.6217 RMSEA 90% Lower Confidence Limit 0.1899 RMSEA 90% Upper Confidence Limit 1.1869 ECVI Estimate 0.9775 ECVI 90% Lower Confidence Limit . ECVI 90% Upper Confidence Limit 2.2444 Probability of Close Fit 0.0237 Bentler's Comparative Fit Index 0.6535 Normal Theory Reweighted LS Chi-Square 9.5588 Akaike's Information Criterion 3.2522 Bozdogan's (1987) CAIC 1.7673 Schwarz's Bayesian Criterion 2.7673 McDonald's (1989) Centrality 0.8376 Bentler & Bonett's (1980) Non-normed Index 0.6535 Bentler & Bonett's (1980) NFI 0.6043 James, Mulaik, & Brett (1982) Parsimonious NFI 0.6043 Z-Test of Wilson & Hilferty (1931) 2.0375 Bollen (1986) Normed Index Rho1 0.6043 Bollen (1988) Non-normed Index Delta2 0.6535 Hoelter's (1983) Critical N 10
The sample size is so small that the p -value should not be taken to be accurate, but to get a small p -value with such a small sample indicates it is possible that the model is seriously deficient . The deficiency could be due to any of the following:
The error variances are not both equal to 0.25.
The error terms are correlated with each other or with the true scores.
The observations are not independent.
There is a disturbance in the linear relation between factrose and factnucl .
The relation between factrose and factnucl is not linear.
The actual distributions are not adequately approximated by the multivariate normal distribution.
A simple and plausible modification to the model is to add a disturbance term or error in the equation to the structural model, as follows.
proc calis data=spleen cov stderr; lineqs sqrtrose=factrose + err_rose, sqrtnucl=factnucl + err_nucl, factrose=beta factnucl + disturb; std err_rose=.25, err_nucl=.25, factnucl=v_factnu, disturb=v_dist; run;
The following parameter estimates are produced.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation factrose = 0.3907*factnucl + 1.0000 disturb Std Err 0.0771 beta t Value 5.0692 Variances of Exogenous Variables Standard Variable Parameter Estimate Error t Value factnucl v_factnu 10.50458 4.58577 2.29 err_rose 0.25000 err_nucl 0.25000 disturb v_dist 0.38153 0.28556 1.34
This model is just identified, so there are no degrees of freedom for the chi-square goodness-of-fittest.
Complicated models are often easier to understand when they are expressed as path diagrams. One advantage of path diagrams over equations is that variances and covariances can be shown directly in the path diagram. Loehlin (1987) provides a detailed discussion of path diagrams.
It is customary to write the names of manifest variables in rectangles and names of latent variables in ovals. The coefficients in each equation are indicated by drawing arrows from the independent variables to the dependent variable. Covariances between exogenous variables are drawn as two-headed arrows. The variance of an exogenous variable can be displayed as a two-headed arrow with both heads pointing to the exogenous variable, since the variance of a variable is the covariance of the variable with itself. Here is a path diagram for the spleen data, explicitly showing all latent variables and variances of exogenous variables.
There is an easier way to draw the path diagram based on McArdle s reticular action model (RAM) (McArdle and McDonald 1984). McArdle uses the convention that a two-headed arrow that points to an endogenous variable actually refers to the error or disturbance term associated with that variable. A two-headed arrow with both heads pointing to the same endogenous variable represents the error or disturbance variance for the equation that determines the endogenous variable; there is no need to draw a separate oval for the error or disturbance term. Similarly, a two-headed arrow connecting two endogenous variables represents the covariance between the error of disturbance terms associated with the endogenous variables. The RAM conventions allow the previous path diagram to be simplified, as follows.
The RAM statement in PROC CALIS provides a simple way to transcribe a path diagram based on the reticular action model. Assign the integers 1, 2, 3, ... to the variables in the order in which they appear in the SAS data set or in the VAR statement, if you use one. Assign subsequent consecutive integers to the latent variables displayed explicitly in the path diagram (excluding the error and disturbance terms implied by two-headed arrows) in any order. Each arrow in the path diagram can then be identified by two numbers indicating the variables connected by the path. The RAM statement consists of a list of descriptions of all the arrows in the path diagram. The descriptions are separated by commas. Each arrow description consists of three or four numbers and, optionally , a name in the following order:
The number of heads the arrow has.
The number of the variable the arrow points to, or either variable if the arrow is two-headed.
The number of the variable the arrow comes from, or the other variable if the arrow is two-headed.
The value of the coefficient or (co)variance that the arrow represents.
A name if the arrow represents a parameter to be estimated, in which case the previous number is taken to be the initial estimate of the parameter. Omit the name if the arrow represents a constant. If you specify a name, the fourth number may be omitted.
The model for the spleen data can be specified with the RAM statement, as follows:
/* 1 sqrtrose */ /* 2 sqrtnucl */ /* 3 factrose */ /* 4 factnucl */ proc calis data=spleen cov stderr method=ml outram=splram1; var sqrtrose sqrtnucl; ram 1 1 3 1, /*sqrtrose <- factrose */ 1 2 4 1, /*sqrtnucl <- factnucl */ 1 3 4 beta, /* factrose <- factnucl */ 2 1 1 .25, /* error variance for sqrtrose */ 2 2 2 .25, /* error variance for sqrtnucl */ 2 3 3 v_dist, /* disturbance variance for factrose */ 2 4 4 v_factnu; /* variance of factnucl */ run;
The resulting output in RAM form is displayed in Figure 13.7.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation RAM Estimates Standard Term Matrix ----Row----- ---Column--- Parameter Estimate Error t Value 1 2 sqrtrose 1 F1 3 . 1.00000 1 2 sqrtnucl 2 F2 4 . 1.00000 1 2 F1 3 F2 4 beta 0.39074 0.07708 5.07 1 3 E1 1 E1 1 . 0.25000 1 3 E2 2 E2 2 . 0.25000 1 3 D1 3 D1 3 v_dist 0.38153 0.28556 1.34 1 3 D2 4 D2 4 v_factnu 10.50458 4.58577 2.29
You can request an output data set containing the model specification by using the OUTRAM= option in the PROC CALIS statement. Names for the latent variables can be specified in a VNAMES statement.
proc calis data=spleen cov stderr method=ml outram=splram1; var sqrtrose sqrtnucl; vnames 1 factrose factnucl, 2 err_rose err_nucl disturb factnucl; ram 1 1 3 1, /*sqrtrose <- factrose */ 1 2 4 1, /*sqrtnucl <- factnucl */ 1 3 4 beta, /* factrose <- factnucl */ 2 1 1 .25, /* error variance for sqrtrose */ 2 2 2 .25, /* error variance for sqrtnucl */ 2 3 3 v_dist, /* disturbance variance for factrose */ 2 4 4 v_factnu; /* variance of factnucl */ run; proc print; run;
The RAM output is displayed in Figure 13.8.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation RAM Estimates Standard Term Matrix ----Row----- ---Column--- Parameter Estimate Error t Value 1 2 sqrtrose 1 factrose 3 . 1.00000 1 2 sqrtnucl 2 factnucl 4 . 1.00000 1 2 factrose 3 factnucl 4 beta 0.39074 0.07708 5.07 1 3 err_rose 1 err_rose 1 . 0.25000 1 3 err_nucl 2 err_nucl 2 . 0.25000 1 3 disturb 3 disturb 3 v_dist 0.38153 0.28556 1.34 1 3 factnucl 4 factnucl 4 v_factnu 10.50458 4.58577 2.29
The OUTRAM= data set contains the RAM model as you specified it in the RAM statement, but it contains the final parameter estimates and standard errors instead of the initial values.
Obs _TYPE_ _NAME_ _MATNR_ _ROW_ _COL_ _ESTIM_ _STDERR_ 1 MODEL _IDE_ 1 2 4 1.0000 0.00000 2 MODEL _A_ 2 4 4 6.0000 2.00000 3 MODEL _P_ 3 4 4 3.0000 0.00000 4 VARNAME sqrtrose 2 . 1 . . 5 VARNAME sqrtnucl 2 . 2 . . 6 VARNAME factrose 2 . 3 . . 7 VARNAME factnucl 2 . 4 . . 8 VARNAME err_rose 3 . 1 . . 9 VARNAME err_nucl 3 . 2 . . 10 VARNAME disturb 3 . 3 . . 11 VARNAME factnucl 3 . 4 . . 12 METHOD ML . . . . . 13 STAT N . . . 12.0000 . 14 STAT FIT . . . 0.0000 . 15 STAT GFI . . . 1.0000 . 16 STAT AGFI . . . . . 17 STAT RMR . . . 0.0000 . 18 STAT PGFI . . . 0.0000 . 19 STAT NPARM . . . 3.0000 . 20 STAT DF . . . 0.0000 . 21 STAT N_ACT . . . 0.0000 . 22 STAT CHISQUAR . . . 0.0000 . 23 STAT P_CHISQ . . . 0.0000 . 24 STAT CHISQNUL . . . 13.2732 . 25 STAT RMSEAEST . . . 0.0000 . 26 STAT RMSEALOB . . . . . 27 STAT RMSEAUPB . . . . . 28 STAT P_CLOSFT . . . . . 29 STAT ECVI_EST . . . 0.7500 . 30 STAT ECVI_LOB . . . . . 31 STAT ECVI_UPB . . . . . 32 STAT COMPFITI . . . 1.0000 . 33 STAT ADJCHISQ . . . . . 34 STAT P_ACHISQ . . . . . 35 STAT RLSCHISQ . . . 0.0000 . 36 STAT AIC . . . 0.0000 . 37 STAT CAIC . . . 0.0000 . 38 STAT SBC . . . 0.0000 . 39 STAT CENTRALI . . . 1.0000 . 40 STAT BB_NONOR . . . . . 41 STAT BB_NORMD . . . 1.0000 . 42 STAT PARSIMON . . . 0.0000 . 43 STAT ZTESTWH . . . . . 44 STAT BOL_RHO1 . . . . . 45 STAT BOL_DEL2 . . . 1.0000 . 46 STAT CNHOELT . . . . . 47 ESTIM 2 1 3 1.0000 0.00000 48 ESTIM 2 2 4 1.0000 0.00000 49 ESTIM beta 2 3 4 0.3907 0.07708 50 ESTIM 3 1 1 0.2500 0.00000 51 ESTIM 3 2 2 0.2500 0.00000 52 ESTIM v_dist 3 3 3 0.3815 0.28556 53 ESTIM v_factnu 3 4 4 10.5046 4.58577
This data set can be used as input to another run of PROC CALIS with the INRAM= option in the PROC CALIS statement. For example, if the iteration limit is exceeded, you can use the RAM data set to start a new run that begins with the final estimates from the last run. Or you can change the data set to add or remove constraints or modify the model in various other ways. The easiest way to change a RAM data set is to use the FSEDIT procedure, but you can also use a DATA step. For example, you could set the variance of the disturbance term to zero, effectively removing the disturbance from the equation, by removing the parameter name v_dist in the _NAME_ variable and setting the value of the estimate to zero in the _ESTIM_ variable:
data splram2(type=ram); set splram1; if _name_='v_dist' then do; _name_=' '; _estim_=0; end; run; proc calis data=spleen inram=splram2 cov stderr; run;
The resulting RAM output is displayed in Figure 13.10.
The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation RAM Estimates Standard Term Matrix ----Row----- ---Column--- Parameter Estimate Error t Value 1 2 sqrtrose 1 factrose 3 . 1.00000 1 2 sqrtnucl 2 factnucl 4 . 1.00000 1 2 factrose 3 factnucl 4 beta 0.40340 0.05078 7.94 1 3 err_rose 1 err_rose 1 . 0.25000 1 3 err_nucl 2 err_nucl 2 . 0.25000 1 3 disturb 3 disturb 3 . 0 1 3 factnucl 4 factnucl 4 v_factnu 10.45846 4.56608 2.29