The multivariate statistical method of Principle Component Analysis is a very useful tool for reducing the number of variables in a data set and for obtaining useful two-dimensional views of a multi-dimensional data set. As explained above, the data matrix consisting of fifteen elements can be considered to exist in fifteen-dimensional space (since this would be the number of dimensions required to simultaneously plot all of the variables against one another). For a data set with multivariate normal distribution, when the data points are all plotted they will form a "cloud" of points which may have an oval to circular cross-section in any particular direction. A three-dimensional version can be pictured as a (flattened-)football-shaped cloud of data points.

A Principal Component Analysis of the data set will determine the perpendicular axes (called eigenvectors) which are defined by the dimensions of the data set. There will be the same number of axes as variables/dimensions; the longest axis is the First Principle Component (PC1), the next major axis is the Second Principle Component (PC2), etc. In the example of a three-dimensional football shape, PC1 is the axis running through the football tip to tip, and PC2 and PC3 are two equal perpendicular axes through the equator of the football. If the football is deflated and flattened a bit, then PC2 and PC3 are no longer equal; PC2 is by definition the longer of the two.

The nature of the Principle Components does not change if the data set is not multivariate normal, or consists of several subgroups of data. PC1 will still be the longest possible axis running through the data points, PC2 will be the longest possible axis perpendicular to PC1, and so forth.

The advantage of defining these Principle Component axes is that the axes can now be used to define planar sections through the data set. If one makes a slice through the cloud of data points using the plane defined by PC1 and PC2 and projects all of the data points onto this plane, then it becomes a two-dimensional representation of the data retaining the maximum variation (and hopefully information) contained in the multivariate data set. In many cases, this is the best two-dimensional representation of the multi-dimensional system. Similarly, if the multi-dimensional data contain multiple separate clusters of points, the plane of PC1 and PC2 will often provide a view of the maximum two-dimensional separation between them. Depending upon the distribution of the data set and the intended goal of the analysis, this will often be the best two-dimensional representation of a set of multi-dimensional data clusters.

Furthermore, each variable in the analyzed data set can be assessed concerning its contribution to the overall distribution of the data set. This is done by correlating the direction of maximum spread of each variable with the direction of each Principle Component axis (eigenvector). If one particular variable has a much larger range of values than others (for example, if it is responsible for stretching out a 3-D sphere of data points into an elongate football-shape), then the direction of maximum spread for this variable will strongly correspond to PC1. A high correlation between PC1 and a variable indicates that the variable is associated with the direction of the maximum amount of variation in the data set. More than one variable may correspond highly with PC1; more than one variable may be having a strong influence on the distribution of the data. Similarly, if the whole data set contains two data clusters and a single variable corresponds highly with PC1, then that variable may be responsible for the separation and unique definition of the two data groups. A strong correlation between a variable and PC2 indicates that the variable is responsible for the next largest variation in the data perpendicular to PC1, and so on.

Conversely, if a variable does not correspond to any PC axis, or corresponds only with high-number PC axes, this usually suggests that the variable has little or no control on the distribution of the data set. Therefore, Principle Component Analysis may often indicate which variables in a data set are important and which ones may be of little consequence. Some of these low-performance variables might therefore be "weeded out" and removed from consideration in order to simplify the overall analyses.

For PCA, the calculations of eigenvectors can be made using either the covariance matrix or the correlation matrix of the data set. The latter is commonly used when different variables in the data set are measured in different units, or if different variables have strongly different variances. Using the correlation matrix recalculates all of the variables so that their variances are equal. This can be a significant concern with geochemical data, since some elements typically have a much broader range of concentrations than others in the samples.

Such is a quick overview of Principle Component Analysis and its potential benefits. PCA was conducted for the Prairie du Chien data set using the SAS procedure princomp.

Out of a total of 55 elements analyzed in the INAA process, only fifteen were consistently present in both standard and unknowns. However, it is quite possible that out of the fifteen elements, only a few provide most of the meaningful information to be found in the geochemical data. Also, some elements may have high degrees of intercorrelation, in essence producing redundant information. Principle Component Analysis is in many cases a good method for addressing these possibilities. The elements which provide little information to the data set are likely to be adding "noise" to the analyses. Therefore it might be desirable to narrow down the number of variables.

The SAS procedure princomp was run using the correlation matrix of the geochemical data set. Part of the output is in the form of the eigenvector/element correlation matrix found in Table 3. As explained above, those elements which correlate highly with the first two or three eigenvectors (Principle Components) are the variables with the greatest variability in the data set. As can be seen in the table, PC1 correlates moderately well with five elements (Ce, Fe, La, Sc, and Sm), and PC2 correlates moderately well with four elements (Al, Cr, Cs, and Eu); two of these nine elements (Al and Eu) also correlate moderately with PC3. Therefore these nine elements were chosen as the best variables to represent the Prairie du Chien data set, and the others were excluded from further Principle Component analyses.

Table 3: Correlation of Principle Component Analysis eigenvectors with elements. Numbers with high absolute values indicate that the corresponding element has a strong influence on the given eigenvector (Principle Component).

With the number of element variables shortened to the nine listed above, the Principle Component Analysis was again run on the chert data set, the output of which can be seen in Tables 4 - 6. Table 4 shows the calculated correlations of elements with each other. Values near one indicate that a pair of elements in the data set are behaving nearly identically. That is, they both increase/decrease the same way when plotted. Conversely, a value near -1 indicates that a pair of elements are behaving in opposite manners – one increases whenever the other decreases, etc. A value near zero indicates that when two elements are plotted they are behaving independently from one another. These number values can also be expressed as percentages by moving the decimal.

It is notable that cerium, lanthanum, and samarium all correlate rather highly with one another: cerium and lanthanum 61%; cerium and samarium 72%; and lanthanum and samarium 80%. Also, lanthanum correlates rather well with scandium, at 62%. This is not surprising, since Ce, La, and Sm are all Rare Earth Elements, and Sc behaves similarly to the Rare Earth Elements in sedimentary processes (McLennan, 1989). This information is useful, since researchers have sometimes found that elements with high correlations may reduce the amount of random "noise" in a data set when their ratios are used as input rather than their individual concentrations (e.g. Aspinall and Feather, 1972). Therefore these four element ratios were later incorporated into some of the analyses rather than the individual element concentrations.

As can be seen in Table 5, PC1 correlates moderately strongly with the elements lanthanum (La), and samarium (Sm); PC2 correlates moderately strongly with chromium (Cr) and cesium (Cs), and has a negative correlation with europium (Eu); PC3 has a strong correlation with aluminum (Al). This indicates that these six elements are most strongly associated with the directions of maximum variance in the data set. It is also notable that PC1 has positive correlations with all the elements, which would indicate that to a certain extent, PC1 is controlled by an overall increase in element concentrations.

The information in Table 6 indicates that PC1 accounts for about 43% of the variation in the chert data set; PC2 accounts for 20%, and the percentages drop off gradually for the rest of the Principle Components. Cumulatively, PC1 and PC2 together account for 62% of the variation in the data set. A two-dimensional view (of the nine-dimensional data set) can be created by projecting all of the data points onto the plane defined by the axes PC1 and PC2. This two-dimensional view will retain 62% of the information from the nine-dimensional plot. As described earlier for Principle Component Analysis in general, this plane derived from the axes PC1 and PC2 will provide a two-dimensional plot retaining the maximum possible amount of information from the multivariate plot. This plot of PC1 versus PC2 can be seen in Figure 7. The scale on the axes of the Principle Component plots are in increments of standard deviations from the variable means (this is done to equalize the scale at which each variable is plotted and compared to the others).

The overall plot for the entire Prairie du Chien data set is very dense and close to having a multivariate normal distribution. That is, it is nearly elliptical in distribution with few outliers. In fact, the overall grouping is so tight that there is a great deal of overlap between the sample groupings from individual sample locations. This is somewhat unfortunate, though not surprising given that they are, after all, derived from essentially the same geologic formation. Had the samples come from multiple formations, they probably would appear as two or more individual groupings, separate or somewhat overlapping. The tight clustering and overlap between individual locations in this case prevents the identification or definition of distinctive provenance sources, and indicates that they will probably not be easily distinguished geochemically by any means.

A second PCA plot was made for the data, using the four element ratios identified above. This was as a test to see if the ratios might remove any noise from the data and reveal otherwise unseen trends. Once again, the samples were plotted according to sample location. The element ratios did not do any better at separating sample locations, but produced an interesting trend in about half of the individual sample plots. The samples started separating into two separate groups within many of the sample locations, both outcrop and stream deposits. When each data point was labeled with its corresponding sample number, however, there was no evident reason for the separation. The groups do not correspond to Shakopee and Oneota, or any other stratigraphic rule; nor do they correspond to any textural or other visible macroscopic feature. It seems that either there is an unknown process influencing the rare earth element ratios throughout the Prairie du Chien, or else there is a quirk of the statistical analytical process which produces these apparent separations.

In any case, there is still no evident means of distinguishing different source regions using PCA. Therefore further methods other than PCA must be employed.

     NEXT     PREVIOUS     TABLE OF CONTENTS

Intro and Background     Fieldwork     Sample Prep     Data Analysis     PCA     Correspondence Analysis    Stepwise DA   

Discriminant Analysis     More PCA     Element Trends     Conclusions     Bibliography     Appendix A: Part 1 Part 2 Part 3 Part 4 Part 5     Appendix B     Appendix C

The views and opinions expressed in this page are strictly those of the page author.
The contents of this page have not been reviewed or approved by the University of Minnesota.