WHAT IS FACTOR ANALYSIS?

Factor analysis is a generic name given to a class of multivariate statistical methods whose primary purpose is to define the underlying structure in a data matrix. Broadly speaking, it addresses the problem of analyzing the structure of the interrelationships ( correlations ) among a large number of variables (e.g., test scores, test items, questionnaire responses) by defining a set of common underlying dimensions, known as factors. With factor analysis, the experimenter can first identify the separate dimensions of the structure and then determine the extent to which each variable is explained by each dimension. Once these dimensions and the explanation of each variable are determined, the two primary uses for factor analysis ” summarization and data reduction ” can be achieved. In summarizing the data, factor analysis derives underlying dimensions that, when interpreted and understood , describe the data in a much smaller number of concepts than the original individual variables. Data reduction can be achieved by calculating scores for each underlying dimension and substituting them for the original variables.

We introduce factor analysis as our first multivariate technique because it can play a unique role in the application of other multivariate techniques. As already discussed, the primary advantage of multivariate techniques is their ability to accommodate multiple variables in an attempt to understand the complex relationships not possible with univariate and bivariate methods. Increasing the number of variables also increases the possibility that the variables are not all uncorrelated and representative of distinct concepts. Instead, groups of variables may be interrelated to the extent that they are all representative of a more general concept. This may be by design, such as the attempt to measure the many facets of personality or store image, or may arise just from the addition of new variables. In either case, the researcher must know how the variables are interrelated to better interpret the results. Finally, if the number of variables is too large or there is a need to better represent a smaller number of concepts rather than many facets, factor analysis can assist in selecting a representative subset of variables or even creating new variables as replacements for the original variables while still retaining their original character.

Factor analysis differs from the dependence techniques discussed in the next section (i.e., multiple regression, discriminant analysis, multivariate analysis of variance, or canonical correlation), in which one or more variables are explicitly considered the criterion or dependent variables and all others are the predictor or independent variables. Factor analysis is an interdependence technique in which all variables are simultaneously considered , each related to all others, and still employing the concept of the variate, the linear composite of variables. In factor analysis, the variates (factors) are formed to maximize their explanation of the entire variable set, not to predict a dependent variable(s). If we were to draw an analogy to dependence techniques, it would be that each of the observed (original) variables is a dependent variable that is a function of some underlying and latent set of factors (dimensions) that are themselves made up of all other variables. Thus, each variable is predicted by all others. Conversely, one can look at each factor (variate) as a dependent variable that is a function of the entire set of observed variables. Either analogy illustrates the differences in purpose between dependence (prediction) and interdependence (identification of structure) techniques.

Factor analytic techniques can achieve their purposes from either an exploratory or confirmatory perspective. There is continued debate concerning the appropriate role for factor analysis. Many researchers consider it only exploratory ” useful in searching for structure among a set of variables or as a data reduction method. From this perspective, factor analytic techniques "take what the data give you" and do not set any a priori constraints on the estimation of components or the number of components to be extracted. For many if not most applications, this use of factor analysis is appropriate. However, in other situations, the experimenter has preconceived thoughts on the actual structure of the data, based on theoretical support or prior research. The experimenter may wish to test hypotheses involving issues such as which variables should be grouped together on a factor or the precise number of factors. In these instances, the experimenter requires that factor analysis take a confirmatory approach ” that is, assess the degree to which the data meet the expected structure.