An Overview of Mixed Effects Models
Amelia Rodelo
Contents:
I. Introduction
II. Contrasting the General Linear Model with the Mixed Effects Model
III. Non-linear Mixed Effects Models
IV. Software
V. Resources
I. Introduction
Mixed Effects Models offer a flexible framework by which to model the sources of variation and correlation that arise from grouped data. This grouping can arise when data-collection is undertaken in a hierarchical manner, when a number of observations are taken on the same observational unit over time, or when observational units are in some other way related, violating assumptions of independence. Mixed Effects Models are seen as especially robust in the analysis of unbalanced data when compared to similar analyses done under the General Linear Model framework (Pinheiro and Bates, 2000). In within-subjects designs (repeated measures or split-plot), subjects on which observations are missing can still be included in the analysis. However, Mixed Effects Models provide an enormous advantage over the General Linear Model in designs where no missing observations are allowed. Also, though the GLM can model situations in which grouped data arise (such as repeated measures ANOVA), its parameter estimations can be problematic and in these situations Mixed Effects Models are preferred (see Garson 2008).
Mixed Effects Model can be used to model both linear and nonlinear relationships between dependent and independent variables. The Mixed Modeling framework can specify a variety of model types including random coefficients models, hierarchical linear models, variance components models, nested models, and split-plot designs. This framework is widely applicable across numerous fields within the behavioral, medical, and environmental sciences.
While under the General Linear Model one can specify multiple random effects terms, one must consider each factor to be either a fixed or random effect. Under the Mixed Effects Modeling approach factors may be considered to have both a fixed and a random component. The Mixed Effects Modeling approach allows the researcher to determine for which terms an additional random component should be included using multi-model inference.
II. Contrasting the General Linear Model with the Mixed
Effects Model
The General Linear Model, in matrix form and taken from Fox (2002), is as follows:
where y = (y1, y2, ..., yn)’ is
the response vector; X is the model matrix, with typical row x’i =
(x1i, x2i,
..., xpi);
β =(β1, β2,
..., βp)’ is
the vector of regression coefficients;
ε
= (ε1, ε2, ..., εn)’ is the vector of errors; Nn represents the n-variable
multivariate-normal distribution; 0 is an n × 1 vector of zeroes; and In is the order - n identity
matrix. This general model specifies one set of random effects under the error
vector
.
A Mixed Effects Model is an extension of the General Linear Model that can specify additional random effects terms (again taken from Fox, 2002):
where yi is the ni x 1 response vector for observations in the ith group, Xi is the ni x p model matrix for the fixed effects for observations in group i, β is the p x 1 vector of fixed-effect coefficients, Zi is the ni x q model matrix for the random effects for observations in group i, bi is the q x 1 vector of random-effect coefficients for group i, εi is the ni x 1 vector of errors for observations in group i, Ψ is the q x q covariance matrix for the random effects, σ2Λi is the ni x ni covariance matrix for the errors in group i. In this framework, multiple sources of random variation can be accounted for under the random effects coefficients term b.
Parameters of the mixed model can be estimated using Maximum Likelihood Estimation (MLE) or Restricted Maximum Likelihood Estimation (RMLE), while the Akaike Information Criteria (AIC) and the Bayesian Information Criteria (BIC) can be used as measures of “goodness of fit” for particular models, where smaller values for both are considered more preferable.
III. Nonlinear Mixed Effects Models
While Linear Mixed Effects Models can be used to express linear relationships between sets of variables, nonlinear models can model mechanistic relationships between independent and dependent variables and can estimate more physically interpretable parameters (Pinheiro and Bates, 2000). The Nonlinear Mixed Effects Model, taken from Bates and Lindstrom (1990), can be represented as follows:
where yij is
the jth response on the ith
individual, xij is a predictor
vector for the jth response on the ith individual, f represents the non-linear
function of the predictor vector and a parameter vector
, and eij is a
normally distribute noise term.
Applicable non-linear functions include those which model a response evolving over time, such as the following function taken from the field of pharmacokinetics. This equation models the outcome of drug concentration at time t using physically interpretable parameters such as fractional rate of absorption, clearance rate, and volume of distribution. This equation was taken from Davidian (2003):
where 
is the fractional rate of absorption, 
is the clearance rate, and V is the volume of distribution.
These mechanistic parameters can be estimated using a
mixture of both fixed and random effects for each term. As seen in the
Nonlinear Mixed Effects Model taken from Bates and Lindstrom, each parameter in
the parameter vector can be defined by both fixed and random effects and can vary
from individual to individual:
where
is a p-vector of fixed population parameters, 
is a q-vector of random effects associated
with individual i,
the matrices 
and 
are design matrices of size r x p and r x q
for the fixed and random effects, respectively, and 
is a
covariance matrix.
The necessity of including random effects to estimate each parameter can be assessed using stepwise addition/deletion of random effects model terms as estimated using MLE or RMLE and AIC and BIC as goodness-of-fit criteria.
IV. Software
Statistical packages for mixed modeling include SPSS, SAS, R, S-Plus, and GenStat, among possible others. For Nonlinear models specifically, the most widely available documentation is in SAS, R, and S-Plus. For R and S-Plus, the NLME library, including extensions for both linear and nonlinear mixed effects modeling, can be taken from the Bates and Pinheiro website (see Useful Resources)
V. Useful Resources, including citations made in this
text:
Davidian, Marie. 2004. “Nonlinear Mixed Effects Models: An Overview and Update” http://www4.stat.ncsu.edu/~davidian/nlmmtalk.pdf Last accessed May 2008
This is an excellent .pdf of a
talk given by Marie Davidian regarding Nonlinear Mixed Effects Models. It
provides a brief introduction as well as three easy-to-understand examples from
biomedicine and ecology, independent of software but using mathematical
definitions for explanation.
Fox, John. 2002. “Linear Mixed Models” http://cran.r-project.org/doc/contrib/Fox-Companion/appendix-mixed-models.pdf Last accessed May 2008.
This online guide is the “official” R documentation for
Linear Mixed Models and provides a succinct 1-page introduction to the general
framework followed by examples in the R language.
Gaccione, Peter, and M.S. Blanchard. 2008. “Nonlinear Mixed Effects Models, a Tool for Analyzing Repeated Measurements; A Brief Tutorial Using SAS Software.” http://ssc.utexas.edu/docs/sashelp/sugi/23/Stats/p228.pdf Last accessed May 2008.
This guide offers a good overview of the fundamental nonlinear mixed effects model and provides an example using SAS software and the NLINMIX macro for Nonlinear mixed effects modeling in SAS.
Galwey, N.W. 2006. “Introduction to Mixed Modeling; Beyond Regression and Analysis of Variance” John Wiley and Sons.
This text provides a basic introduction to Mixed Modeling using contrasts to the General Linear Model as a teaching framework; it provides numerous examples using regression and ANOVA under both the Mixed Modeling framework and the General Linear Model framework. Recommended for those who want a text-based, non-mathematical introduction to mixed modeling.
Garson, G. David. 2008. “Linear Mixed Models: Random Effects, Hierarchical Linear, Multilevel, Random Coefficients, and Repeated Measures Models” http://www2.chass.ncsu.edu/garson/pa765/multilevel.htm Last accessed May 2008.
This online guide provides a lengthy textual overview of the family of mixed effects models as well as directions for using these various types of Mixed Models in SPSS.
Lindstrom, Mary J. and Douglas M. Bates. 1990. Nonlinear mixed effects models for repeated measures data. Biometrics. 46:3 pp 673-687
This is a very
technical explanation of the Nonlinear Mixed Effects Model, though the first
few pages give straightforward definitions of the basic model.
Pinheiro, Jose C. and Douglas M. Bates. 2000. “Mixed Effects Models
in S and S-Plus”
This text is useful for those who have a conceptual idea of the Mixed Modeling Framework and want an example-based introduction to its underlying mathematical theory. It provides in depth explanations of Linear and Nonlinear Mixed Effects Models using examples in the S language.
Pinheiro, Jose C. and Douglas M. Bates. 2001. “NLME: Software for Mixed Effects Models”. http://stat.bell-labs.com/NLME/index.html.
Last accessed May 2008.
This website can be used to download the NLME packages in S authored by Bates and Pinheiro (for R, these packages can be downloaded at R software website: www.r-project.org).