How Does that State Space Model Fit? Extending Structural Equation Model Fit Indices to State Space Models using RMSEAD

Google map

Researchers are increasingly interested in modeling within-person processes using intensive longitudinal data. One prominent approach is the discrete-time state-space model (SSM), which extends structural equation modeling (SEM) to capture time-lagged dependencies. However, standard SEM-based fit indices do not fully address issues of fit when data have temporal dependencies because of the difficulty of fixing the saturated model. We propose an adaptation of the Root Mean Square Error of Approximation Difference (RMSEAD) as an absolute fit measure for SSMs, and evaluate its performance under three SSMs thought to be sufficiently general and complex to serve as the saturated models in the development of RMSEAD.

We conducted simulations to compare the performance of the RMSEAD and difference of the Bayesian Information Criterion (BICD) by varying (1) model complexity, (2) sample size, (3) coefficient magnitude, (4) degree of model misspecification, and (5) saturated models, in univariate and multivariate settings. RMSEAD offers unique advantages in comparison to the BICD in identifying models of reasonable fit. Comparisons to other misspecified alternative models (e.g., white noise model and model with misspecified intercept) demonstrated that RMSEAD helped clarify questions of "adequate" fit even in cases where the BICD would select a highly misspecified model in comparing bad to worse models. Consistent with known properties of RMSEA, model complexity and sample size significantly impacted the efficacy of "standard" RMSEAD cut-off values used for determining approximately adequate fit. However, the proposed RMSEAD can provide a measure of absolute fit in situations when a single SSM model is considered, or to narrow down the subset of candidate longitudinal models that constitute acceptable fit for further downstream selection. An empirical data set with ecological momentary assessments of 40 individuals' mood and anxiety symptoms over 30 days is used to address the adequacy of popular dynamic network models in accounting for the dynamics of ILD.