Exploratory Factor Analysis

Exploratory factor analysis discovers the latent factor structure of a set of observed variables — how many underlying dimensions exist and which items load on which dimension — as a basis for scale construction or instrument refinement.

What it is

Exploratory factor analysis (EFA) models a set of observed variables as linear combinations of a smaller number of unobserved (latent) factors plus item-specific error. Unlike PCA, which maximises explained variance, EFA explicitly models the distinction between shared variance (attributable to the common factors) and unique variance (attributable to item-specific factors and error). The goal is to discover the latent structure underlying the data: how many factors are present, which items belong to each factor, and how strongly each item reflects its factor.

EFA is the standard first step in scale development — when a researcher has written a battery of items to measure one or more psychological constructs and wants to empirically verify the factor structure before computing subscale scores. It is “exploratory” because no structure is imposed in advance; the factor solution is determined by the data.

When to use it

EFA is appropriate when: a multi-item battery has been administered; the researcher has a theory that the items reflect one or more underlying constructs; and the goal is to determine empirically how many factors are present and which items load on which factor.

In development economics field research, EFA is commonly used to validate psychological and non-cognitive skill scales — self-efficacy, locus of control, empowerment indices — before using them as outcomes or controls. Malhotra, Schuler and Boender (2002) use factor analysis to develop a women’s empowerment index for India, identifying distinct dimensions of mobility, economic participation, and decision-making autonomy. Alkire (2005) uses factor analysis to structure multidimensional poverty measures.

EFA is less appropriate when the number and structure of factors is already specified from prior work — confirmatory factor analysis is used in that case — or when the goal is simply to reduce many variables to a single composite (PCA is more appropriate and more transparent).

How it works

Extraction. The most common extraction method is maximum likelihood (ML) when variables are approximately normally distributed, or principal axis factoring (PAF) when normality cannot be assumed. Both identify the factor loadings — the correlations between each observed item and each latent factor — that best reproduce the observed correlation matrix.

Number of factors. Several criteria inform how many factors to retain:

Scree plot: plot eigenvalues against factor number; retain factors above the “elbow.”
Kaiser’s rule: retain factors with eigenvalue > 1 (overly liberal; often extracts too many factors).
Parallel analysis: compare observed eigenvalues to those from randomly generated data of the same size; retain factors where observed eigenvalue exceeds the random expectation. This is the most defensible criterion.
Interpretability: the retained solution should yield factors that are substantively interpretable.

Rotation. Initial factor solutions are rotated to improve interpretability. Orthogonal rotation (Varimax) constrains factors to be uncorrelated and is appropriate when factors are theorised to be independent dimensions. Oblique rotation (Promax, Oblimin) allows correlated factors and is more realistic when factors are expected to be related (e.g., subscales of the same broad construct). Oblique rotation is generally preferred unless there is strong theoretical reason to expect orthogonal dimensions.

Factor loadings and the simple structure ideal. A clean factor solution has each item loading highly on one factor and near-zero on all others. Items with high cross-loadings (loading substantially on two or more factors) are candidates for removal or revision, as they do not cleanly differentiate the constructs. Items with low loadings on all factors contribute little to the factor structure and are candidates for removal.

Key decisions

Sample size. Stable factor solutions require adequate sample size. Rules of thumb suggest at least 5–10 observations per item, and at least 200 observations overall. With smaller samples, factor solutions can be sensitive to minor variation in the data. For scale development with a 20-item battery, a sample of at least 200 is advisable; 500 is better.

Continuous vs. ordinal items. Standard EFA assumes continuous, normally distributed items. For Likert-scale items (1–5 or 1–7 scale) or binary items, polychoric correlations (for ordinal) or tetrachoric correlations (for binary) should be used as input to the factor analysis, rather than Pearson correlations. Using Pearson correlations on ordinal items underestimates factor loadings when response distributions are skewed.

Item refinement. EFA is typically an iterative process. Items with low communality (variance explained by the common factors), high cross-loadings, or poor face validity are removed or revised, and the analysis is repeated. This process should be guided by both statistical criteria and substantive judgment about whether each item captures the intended construct.

Calibration sample. EFA should be run on a development sample, not the full analytic sample. Calibrating the factor structure on the same data used for outcome analysis inflates apparent fit and risks capitalising on chance. In longitudinal studies, EFA on a baseline subsample and confirmatory factor analysis on the full analytic sample is the gold standard.

Caveats & common mistakes

EFA is not confirmatory. Because EFA searches for the best-fitting structure in the data, it will always find a factor solution. High factor loadings and adequate fit statistics do not confirm that the structure is theoretically valid or replicable. The solution must be replicated in a new sample (ideally via CFA) before being used as the basis for a measurement instrument.

Misuse of Kaiser’s rule. Retaining factors with eigenvalue > 1 consistently extracts too many factors. Parallel analysis is more accurate and should be used instead. Many empirical papers use Kaiser’s rule because it is the default in common software; this results in over-factored solutions with poorly interpretable extra factors.

Rotation indeterminacy. Factor analysis solutions are not unique — any rotation of the factor matrix fits the data equally well. The choice of rotation affects the factor structure reported. Oblique rotation is generally the more defensible choice; the choice and the rationale should be reported.

Combining EFA and regression on the same data. If factor scores from an EFA run on the full sample are then used as predictors or outcomes in regression analysis on the same sample, standard errors are incorrect and the factor structure is effectively optimised to explain the outcome. Split-sample analysis or pre-registration of the factor structure avoids this.

Analysis Guide

# item_1 to item_10: Likert or continuous scale items
import pandas as pd
import numpy as np
from factor_analyzer import FactorAnalyzer
from factor_analyzer.factor_analyzer import calculate_kmo, calculate_bartlett_sphericity

item_cols = [f"item_{i}" for i in range(1, 11)]

# 1. KMO and Bartlett — KMO > 0.60 indicates the correlation matrix has enough common variance to support EFA; a significant Bartlett test (p < 0.05) confirms the matrix is not an identity, i.e., items are correlated enough to factor
kmo_all, kmo_model = calculate_kmo(df[item_cols])
chi2, p = calculate_bartlett_sphericity(df[item_cols])

# 2. Parallel analysis — compares observed eigenvalues against eigenvalues from randomly generated data of the same dimensions; retain only factors where observed exceeds the random benchmark; more accurate than Kaiser's eigenvalue > 1 rule
fa_init = FactorAnalyzer(rotation=None, n_factors=len(item_cols)).fit(df[item_cols])
obs_ev, _ = fa_init.get_eigenvalues()
rand_ev = np.mean([np.linalg.eigvalsh(np.corrcoef(np.random.normal(size=df[item_cols].shape), rowvar=False))[::-1]
                 for _ in range(100)], axis=0)
n_factors = int((obs_ev > rand_ev).sum())

# 3. Run EFA with maximum likelihood extraction and oblique (Promax) rotation — ML is preferred when items are approximately normal; oblique rotation allows factors to correlate, which is more realistic for psychological constructs that are theoretically related
fa = FactorAnalyzer(n_factors=n_factors, rotation="promax", method="ml").fit(df[item_cols])

# 4. Inspect factor loadings — items loading >= 0.40 on a factor are meaningfully associated; items with cross-loadings >= 0.30 do not cleanly differentiate the constructs and are candidates for removal
loadings = pd.DataFrame(fa.loadings_, index=item_cols)
print(loadings.round(2))

# 5. Communalities — proportion of each item's variance explained by the retained factors; communalities below 0.20 indicate items weakly tied to the factor structure and candidates for removal
print(pd.Series(fa.get_communalities(), index=item_cols))

# 6. Factor scores for each respondent — person-level estimates of position on each latent dimension, for use in subsequent analysis
scores = pd.DataFrame(fa.transform(df[item_cols]),
                    columns=[f"factor{i+1}" for i in range(n_factors)])
df = pd.concat([df, scores], axis=1)

# Note: for polychoric correlations on Likert items, pass the polychoric matrix from
# factor_analyzer.utils.polychoric_correlations as is_corr_matrix=True to FactorAnalyzer.

library(psych)
item_cols <- paste0("item_", 1:10)

# 1. Compute polychoric correlation matrix — for ordered Likert items, polychoric correlations are more appropriate than Pearson because they account for the ordinal (not continuous) nature of responses and avoid attenuation from skewed distributions
poly_cor <- polychoric(df[, item_cols])$rho

# 2. Parallel analysis — determines number of factors to retain by comparing observed eigenvalues against eigenvalues from random data of the same size; retain factors where observed > random; more accurate than Kaiser's eigenvalue > 1 rule
fa.parallel(poly_cor, n.obs = nrow(df), fm = "ml", fa = "fa")

# 3. Run EFA with oblique rotation — oblimin allows factors to correlate, which is more appropriate than varimax when factors are theoretically related subscales of the same broad construct; nfactors set based on parallel analysis result
efa_result <- fa(poly_cor, nfactors = 2, rotate = "oblimin",
               fm = "ml", n.obs = nrow(df))
print(efa_result$loadings, cutoff = 0.3)   # suppress loadings below 0.30 for readability

# 4. Factor diagram — visualises which items load on which factors; useful for assessing whether the solution has a clean simple structure
fa.diagram(efa_result)

# 5. Extract factor scores for each respondent using raw data — regression scoring produces least-squares estimates of each person's position on each latent factor
efa_scores <- fa(df[, item_cols], nfactors = 2, rotate = "oblimin", fm = "ml",
               scores = "regression")
df$factor1 <- efa_scores$scores[, 1]
df$factor2 <- efa_scores$scores[, 2]

Reading the output

Factor loadings ≥ 0.40 indicate a meaningful relationship between the item and the factor; loadings between 0.30 and 0.40 are borderline. Items with all loadings below 0.30 have low communality and contribute little to the factor structure — consider removing them.
Cross-loadings ≥ 0.30 on a second factor indicate the item does not cleanly differentiate the two constructs; items with high cross-loadings are candidates for removal or revision.
Parallel analysis results show how many factors have eigenvalues exceeding those from random data. Retain only factors where the observed eigenvalue exceeds the parallel analysis threshold — this is more conservative than Kaiser’s rule (eigenvalue > 1) and generally more accurate.
Communalities (from fa.get_communalities() / efa_result$communality) below 0.20 indicate that the common factors explain very little of that item’s variance; these items are weakly tied to the shared construct.
If the factor correlation from an oblique solution (printed in R as $Phi) exceeds 0.50, the factors are highly related — consider whether a single-factor solution is more parsimonious and theoretically defensible.

References

Alkire, S. (2005). Subjective quantitative studies of human agency. Social Indicators Research, 74(1), 217–260. https://doi.org/10.1007/s11205-005-6525-0

Comrey, A. L., & Lee, H. B. (1992). A First Course in Factor Analysis (2nd ed.). Erlbaum.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272

Malhotra, A., Schuler, S. R., & Boender, C. (2002). Measuring women’s empowerment as a variable in international development. Background paper prepared for the World Bank Workshop on Poverty and Gender. World Bank.