This tutorial will walk through the examples provided in Chapter 7 of Bolger & Laurenceau’s Intensive Longitudinal Methods (2013). The book chapter contains MPlus, SPSS, and SAS code, so we will translate those methods into R. The data set used in this tutorial consists of 50 individuals measured on 4 items over 10 time points.
Generalizability Theory (or G Theory) can answer the question, “can within-subject changes be measured reliably?” In order to answer this question, we must specify the dimensions of generalizability. In this example, the dimensions are time points, persons, items.
To perform this analysis we will use an ANOVA with random effects for each of the specified dimensions.
Loading liraries used in this script.
library(lme4) #for multilevel models
library(tidyr) #for dat management We use the example data from Bolger & Laurenceau (2013) chapter 7.
Loading the data.
filepath <- "https://quantdev.ssri.psu.edu/sites/qdev/files/psychometrics.csv"
psychometrics <- read.csv(filepath)
head(psychometrics)## person time item y
## 1 301 1 1 2
## 2 301 1 2 2
## 3 301 1 3 3
## 4 301 1 4 4
## 5 301 2 1 2
## 6 301 2 2 3Restructuring the item variable so that it is categorical using the factor() function.
psychometrics$item <- factor(psychometrics$item)The lme4 package contains the function lmer() which fits a liner mixed-effects model to a given data set. The summary() of our model will show us the effects of each dimension of generalizability. This is an intercept-only model (ANOVA with random effects) so that we can understand the sources of variability between the dimensions.
model1 <- lmer(y ~ 1 +
(1|person) +
(1|time) +
(1|item) +
(1|person:time) +
(1|person:item) +
(1|time:item),
data=psychometrics)
summary(model1)## Linear mixed model fit by REML ['lmerMod']
## Formula:
## y ~ 1 + (1 | person) + (1 | time) + (1 | item) + (1 | person:time) +
## (1 | person:item) + (1 | time:item)
## Data: psychometrics
##
## REML criterion at convergence: 4046.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2721 -0.5311 -0.0105 0.5044 3.8613
##
## Random effects:
## Groups Name Variance Std.Dev.
## person:time (Intercept) 2.553e-01 5.053e-01
## person:item (Intercept) 1.901e-01 4.360e-01
## person (Intercept) 3.619e-01 6.016e-01
## time:item (Intercept) 4.985e-03 7.060e-02
## time (Intercept) 7.400e-09 8.602e-05
## item (Intercept) 4.854e-02 2.203e-01
## Residual 2.995e-01 5.473e-01
## Number of obs: 1802, groups:
## person:time, 455; person:item, 200; person, 50; time:item, 40; time, 10; item, 4
##
## Fixed effects:
## Estimate Std. Error t value
## (Intercept) 2.4340 0.1456 16.72Using the VarCorr() function, we can extract and save each variance value from the summary table.
(personTime <- VarCorr(model1)[[1]][1,1]) #person:time ## [1] 0.2552794(personItem <- VarCorr(model1)[[2]][1,1]) #person:item ## [1] 0.1900742(person <- VarCorr(model1)[[3]][1,1]) #person ## [1] 0.3618975(timeItem <- VarCorr(model1)[[4]][1,1]) #time:item ## [1] 0.004984693(time <- VarCorr(model1)[[5]][1,1]) #time ## [1] 7.399594e-09(item <- VarCorr(model1)[[6]][1,1]) #item ## [1] 0.04854308(residual <- sigma(model1)^2) #residual## [1] 0.2995418Back to our initial question: Are there reliable within-person differences in change over time?
We’ll use the following formula to calculate the reliability coefficient: \(Rc = \frac{\sigma^2_{person:time}}{(\sigma^2_{person:time} + (\sigma^2_{person:time:item} + \sigma^2_{error})/k)}\)
where \(k\) refers to the number of items. In our case, \(k = 4\).
We cannot tease apart the \(\sigma^2_{person:time:item}\) from the \(\sigma^2_{error}\), so we will use the residual error term.
k <- 4
(Rc <- personTime/(personTime + residual/k))## [1] 0.7731877This coefficient represents the degree to which these repeated measures are adequate and systematic. Using the same interpretation rules of thumb as Cronbach’s alpha, we can determine that four items can capture within person change reliably, \(Rc = 0.77\).
Another coefficent of interest is \(R_{1F}\), which is calculated as
\(R_{1F} = \frac{\sigma^2_{person} + [\sigma^2_{person:item}/k]}{\sigma^2_{person} + [\sigma^2_{person:item}/k] + [\sigma^2_{error}/k]}\)
where \(k\) refers to the number of items. In our case, \(k = 4\).
Our calcuation is
k <- 4
(R1f <- (person + (personItem/k))/(person + (personItem/k) + (residual/k)))## [1] 0.8453743The reliability coefficient is the expected between-person reliability estimate for one fixed day, a kind of average of day-specific Cronbach’s alphas across occasions. Using the same interpretation rules of thumb as Cronbach’s alpha, we can determine that four items can capture between-person differences on any given day reliably, \(R_{1F} = 0.85\).
The book chapter contains MPlus code to calculate the same metrics. To see this example in SAS, MPlus, SPSS, and another R example, please visit [http://www.intensivelongitudinal.com/ch7/ch7index.html]. The lavaan package in R can perform structure equation modeling. In the current version on CRAN, it cannot perform multilevel structural equation modeling. However, that function is on the developers list.