Overview

This tutorial will walk through the examples provided in Chapter 7 of Bolger & Laurenceau’s Intensive Longitudinal Methods (2013). The book chapter contains MPlus, SPSS, and SAS code, so we will translate those methods into R. The data set used in this tutorial consists of 50 individuals measured on 4 items over 10 time points.

Generalizability Theory

Generalizability Theory (or G Theory) can answer the question, “can within-subject changes be measured reliably?” In order to answer this question, we must specify the dimensions of generalizability. In this example, the dimensions are time points, persons, items.

To perform this analysis we will use an ANOVA with random effects for each of the specified dimensions.

library(lme4)
library(tidyr)
filepath <- "https://quantdev.ssri.psu.edu/sites/qdev/files/psychometrics.csv"
psychometrics <- read.csv(filepath)
head(psychometrics)
##   person time item y
## 1    301    1    1 2
## 2    301    1    2 2
## 3    301    1    3 3
## 4    301    1    4 4
## 5    301    2    1 2
## 6    301    2    2 3

Restructuring the item variable so that it is categorical using the factor() function.

psychometrics$item <- factor(psychometrics$item)

The lme4 package contains the function lmer() which fits a liner mixed-effects model to a given data set. The summary() of our model will show us the effects of each dimension of generalizability. This is an intercept-only model (ANOVA with random effects) so that we can understand the sources of variability between the dimensions.

model1 <- lmer(y ~  1 + 
                 (1|person) + 
                 (1|time) + 
                 (1|item) + 
                 (1|person:time) + 
                 (1|person:item) + 
                 (1|time:item), 
               data=psychometrics)
summary(model1)
## Linear mixed model fit by REML ['lmerMod']
## Formula: 
## y ~ 1 + (1 | person) + (1 | time) + (1 | item) + (1 | person:time) +  
##     (1 | person:item) + (1 | time:item)
##    Data: psychometrics
## 
## REML criterion at convergence: 4046.2
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.2721 -0.5310 -0.0105  0.5044  3.8614 
## 
## Random effects:
##  Groups      Name        Variance Std.Dev.
##  person:time (Intercept) 0.255269 0.50524 
##  person:item (Intercept) 0.190110 0.43602 
##  person      (Intercept) 0.361788 0.60149 
##  time:item   (Intercept) 0.004983 0.07059 
##  time        (Intercept) 0.000000 0.00000 
##  item        (Intercept) 0.048591 0.22043 
##  Residual                0.299542 0.54730 
## Number of obs: 1802, groups:  
## person:time, 455; person:item, 200; person, 50; time:item, 40; time, 10; item, 4
## 
## Fixed effects:
##             Estimate Std. Error t value
## (Intercept)   2.4340     0.1456   16.71

Using the VarCorr() function, we can extract and save each variance value from the summary table.

(personTime <- VarCorr(model1)[[1]][1,1]) #person:time 
## [1] 0.2552693
(personItem <- VarCorr(model1)[[2]][1,1]) #person:item 
## [1] 0.1901105
(person <- VarCorr(model1)[[3]][1,1]) #person 
## [1] 0.3617882
(timeItem <- VarCorr(model1)[[4]][1,1]) #time:item 
## [1] 0.004982732
(time <- VarCorr(model1)[[5]][1,1]) #time 
## [1] 0
(item <- VarCorr(model1)[[6]][1,1]) #item 
## [1] 0.04859091
(residual <- sigma(model1)^2) #residual
## [1] 0.2995422

Back to our initial question: Are there reliable within-person differences in change over time?

We’ll use the following formula to calculate the reliability coefficient: \(Rc = \frac{\sigma^2_{person:item}}{(\sigma^2_{person:item} + (\sigma^2_{person:item:time} + \sigma^2_{error})/k)}\)

k refers to the number of items. In our case, \(k = 4\).

We cannot tease apart the \(\sigma^2_{person:item:time}\) from the \(\sigma^2_{error}\), so we will use the residual error term.

k <- 4
(Rc <- personTime/(personTime + residual/k))
## [1] 0.7731805

This coefficient represents the degree to which these repeated measures are adequate and systematic. Using the same interpretation rules of thumb as Cronbach’s alpha, we can determine that four items can capture within person change reliably, \(Rc = 0.77\).

The book chapter contains MPlus code to calculate the same metrics. To see this example in SAS, MPlus, SPSS, and another R example, please visit [http://www.intensivelongitudinal.com/ch7/ch7index.html]. The lavaan package in R can perform structure equation modeling. In the current version on CRAN, it cannot perform multilevel structural equation modeling. However, that function is coming later in 2018.