Welcome to the second part of our four-part blog series aimed at providing you with a brief introduction to linear mixed models. In Part 2, we’ll discuss how to interpret the random model, how to compare the fit of different random models and what BLUPs are.
Understanding the random model
Variance and covariance
For a simple random model with uncorrelated effects, the variance components associated with the random terms generate a variance-covariance matrix for the observations in the data set, where:
- the variance of an observation is equal to the sum of the variance components plus the residual variance. (Note, the variance of the fixed effects is zero, and the variance of each random effect is equal to its variance component.)
- the covariance between any two observations is the sum of the variance components of random effects held in common across the two observations.
For the spring wheat trial described in Part 1, with Genotype fitted as a fixed term and Rep and Rep.Subblock effects fitted as random terms, the total variance of an observation is just the sum of the replicate variance component, the sub-block variance component and the residual variance. i.e.,
where:
- is the observed response on the th plot within the th sub-block within the th replicate
- is the random effect associated with the th replicate with variance component
- is the random effect associated with the th sub-block in the ith replicate with variance component
- is the random deviation for the th plot within the th sub-block within the ith replicate, with residual variance
The covariance between yields from two plots in the same sub-block is then the sum of the replicate and sub-block variance components:
Similarly, the covariance between yields from two plots in the same replicate but different sub-blocks is equal to the replicate variance component, and the covariance between two plots in different replicates is zero, as they have no random terms in common. The variance model generated by the design thus implies that measurements from plots within the same sub-block are more highly correlated than those within the same replicate but in different sub-blocks, and that there is no correlation between measurements from separate replicates. Other patterns of covariance can be generated by using different random terms or correlated random effects.
Negative Variance Components
Whilst the definition of the variance components as variances of the random effects requires that the variance components are positive, in fact, the variance structure requires only that the total variance is positive and, more generally, that the variance of any linear combination of observations is also positive (this property is known as positive-definiteness). In general, as we use random terms to reflect structure, we expect that observations with random effects in common will be more similar than observations without, and so generally variance components are expected to be positive. However, in some circumstances, it is natural to allow variance components to be negative. For example, in field experiments, blocks are laid out on areas of ground thought to be reasonably homogeneous with respect to fertility and other trends. If a mistake is made, so that blocks are laid out in the wrong direction, then plots within the same block may be less alike than plots in different blocks, and this can only be modelled by using a negative variance component for blocks. Furthermore, even if a variance component is positive, it is possible that it may be estimated as a negative value, due to sampling variability. For example, a variance component with a true value equal to zero has a 50% chance of being estimated as a negative value.
Variance Components with Small Standard Errors
Estimates of variance components are presented with their standard errors (SEs). In many cases, some of the estimated variance components are small compared to their SEs, and it might be natural to think of dropping these terms from the model. We advise against this course for two distinct reasons:
- As SEs for variance components depend on an asymptotic approximation, they are only reliable for testing when there is a large amount of information contributing to the estimate. A better approach is the use of likelihood ratio tests (LRTs), which we’ll discuss below.
- Some of the random terms will have been included to describe the randomization structure of the design. This structure is a property of the design and is used to get appropriate degrees of freedom (df) for approximate F tests: the omission of some terms means that the random model is no longer serving this purpose.
Comparing Random Models
Change in Deviance
When using REML estimation, the log-residual likelihood value (ℓR) or deviance (D) can be used to compare nested models that have different random terms but the same set of fixed terms. Note, the deviance is calculated by omitting some constant terms (with respect to the variance parameters) from the log-residual likelihood, then multiplying by -2. A random model that fits well will have a relatively high value of ℓR and hence a relatively low value of the deviance but, because of the omission of the constant terms, the absolute scale of the deviance is arbitrary and it may even be negative.
Two nested random models, with the same fixed terms, can be compared using the difference between their deviances. Let’s denote the larger model as M1 with deviance D1, and drop one or more random terms to obtain the smaller model, denoted as M0 with deviance D0. Now, we wish to test the null hypothesis that the larger model gives no improvement over the smaller model. If the variance parameters for the dropped terms are unconstrained, then under the null hypothesis, the change in deviance, defined as D0–D1, is asymptotically distributed as a chi-square variable with degrees of freedom (df) equal to the number of variance parameters in the dropped terms. For example, if the smaller model is constructed by dropping a single random term associated with one variance parameter (i.e., its variance component) from the random model, then the change in deviance has a chi-square distribution on 1 df.
The situation becomes more complicated when one or more of the variance parameters are constrained. For example, when a random term whose variance component is constrained to be positive is dropped, the asymptotic distribution of the change in deviance is a 50:50 mixture of a chi-square distributions with 0 and 1 df.
Information Criteria
Information criteria can be used to compare linear mixed models with different random models but the same fixed terms. However, unlike the change in deviance, there is no requirement for the random models to be nested. The Akaike and Schwarz information criteria (AIC and SIC, respectively) are defined as:
where t is the number of variance parameters in the model, n is the number of observations, and p is the number of independent fixed parameters. SIC is also known as the Bayesian Information Criterion (BIC). Both AIC and SIC are based on the deviance plus a penalty based on the number of variance parameters estimated and, in the case of SIC, the number of observations used in the log-residual likelihood. The penalty is intended to offset the improvement in likelihood against the lack of parsimony implied by adding more parameters into the model. Using this definition, smaller values of AIC or SIC indicate a better fit of the variance model. The SIC tends to be more conservative than AIC, i.e., to prefer models with fewer variance parameters.
Predictors of Random Effects (BLUPs)
In Part 1 we learned that the parameters of the linear mixed model are the fixed effects and the variance parameters, but we wrote down our models in terms of fixed and random effects. Let’s resolve this apparent contradiction and discuss the status of the random effects.
The linear mixed model can be written in two forms: conditional or marginal.
Conditional model: The model is written in terms of both the fixed and random effects (as in Part 1). Thus, the expected value of the observations depends (i.e., is conditional) on the random effects.
Marginal model: The model is written in terms of the fixed effects and the variance-covariance matrix generated by the random terms. As this model is obtained by integrating over the population of random effects, the expected value of the observations is determined by the fixed terms alone.
Estimation takes place on the marginal model, whose parameters are the fixed effects and the variance parameters. However, we are still often interested in the values of the random effects, and so would like to obtain an estimate of their values. Although clearly this only makes sense when the associated variance component is positive, as otherwise the random effects cannot be defined. Because the random effects are not true parameters, we obtain predictors, rather than estimates, of their values. These predictors are often called BLUPs, which is an acronym for best linear unbiased predictors. In this context, the label ‘unbiased’ can be slightly misleading, as it means that the expected value of the predictor is equal to the expected value of the population, which is zero. This does not mean that the expected value of the predictor is equal to the true random effect. In fact, the predictors are biased towards zero, a property known as shrinkage.
The adjective ‘best’ means that these predictors have minimum mean squared error (defined as variance plus squared bias), conditional on the estimated variance parameters. This property of minimum mean squared error is attractive where accuracy in prediction is more important than unbiasedness, and is sometimes used as a justification for assigning terms to the random rather than fixed model, particularly in the context of variety evaluation.
Now read part 3