Appendix 1: Review of Mathematical Fundamentals

A1.1 Matrix Algebra

A matrix is a rectangular array of numbers. These numbers are organized into rows and columns. A matrix X of n rows and m columns is said to be of order n × m. We will represent this matrix as follows:

The transpose of X is a new matrix X^T derived from X by interchanging the rows and columns of X. Thus,

We can form the sum of two matrices X and Y if they are of the same order. In such a case, a new matrix Z containing the sum is formed as follows:

The following shorthand notation will be used to simplify this operation:

Z = X + Y = (x_ij) + (y_ij) = (x_ij + y_ij) for all i and j

The product of two matrices is defined if the matrices are conformable. The matrices X and Y are conformable if the number of columns of X equals the number of rows of Y. Thus, if X is of order n × m and Y is of order m × p and they are conformable, then the product is defined to be:

A matrix is a square matrix if it has an equal number of rows and columns. A square n × n matrix X is a symmetric matrix if x_ij = x_ji, for all i, j = l,2,...,n. The trace of a square matrix X is the sum of the diagonal elements of X. Thus, the trace of X is defined by:

It follows that:

A1.1.1 Determinants

To simplify the discussion of determinants, a brief discussion of nomenclature is necessary concerning the notion of permutations. If we take the set consisting of the first four counting numbers, {1,2,3,4} , there are 4! or 24 different ways that these four number can be ordered. Let J represent a vector containing a permutation of these four numbers, where J = < j₁, j₂, j₃, j₄ >. Thus, if j₁ has the value of 3, then j₂ can only assume one of the values 1, 2, or 4. Let t represent the number of transpositions to convert the set of integers < 1,2,3,4 > to the new permutation < j₁, j₂, j₃, j₄ >. Let ε^t = +1 if the number of transformations is even and ε^t = -1 if the number of transformations is odd.

The determinant of the matrix X is denoted by |X|. It can be formed from the elements of X by the following computation:

where n is the number of rows and columns in the matrix. In the simplest case, where we have a 2 × 2 matrix, |X| = x₁₁x₂₂ - x₁₂x₂₁.

The cofactor X_ji of an element x_ji of X is the product of (-1)^i+j and the determinant of reduced matrix of X whose i^th row and j^th column have been deleted. We can then represent the determinant of X as:

where i can be any value i = 1,2,...,n.

A1.1.2 Matrix Inverses

If the determinant of X exists, that is |X| ≠ 0, then there is a unique matrix Y such that XY = I, the identity matrix. The matrix Y is the inverse of matrix X. It is sometimes represented as X^-1. Each element y_ij of the inverse matrix Y is derived from X as:

where X_ji is a cofactor of X.

A1.1.3 Eigenproblems

The eigenvalues of a square matrix X are the roots of the determinantal equation:

e(λ) = |X-λI| = 0

The n roots of e(λ) are λ₁,λ₂,...,λ_n. For each i = 1,2,...,n, observe that |X-λ_iI| = 0. Thus, X-λ_iI is singular and there exists a nonzero vector e_i with the property that:

Xe_i = λ_ie_i

Any vector with this property is called an eigenvector of X for the eigenvalue λ_i.

A1.1.4 Spectral Decomposition

Any symmetric matrix X, such as a variance-covariance matrix or a matrix of correlation coefficients, can be written as:

X = E∧E'

where ∧ is a diagonal matrix of the eigenvalues of X and E is an orthogonal matrix whose columns are the associated eigenvectors of each of the eigenvalues in ∧. This result is called the spectral decomposition of X. If the eigenvalues of a variance-covariance matrix X are extracted from the largest eigenvalue to the smallest, the resulting spectral decomposition will produce the principal components of X.