Asymptotic Covariance

Asymptotic covariance are derived based on the estimate of β.

Following the definition in details in model fitting, we have

β̂ = (X^⊤WX)⁻¹X^⊤WY

cov(β̂) = (X^⊤WX)⁻¹X^⊤Wcov(ϵ)WX(X^⊤WX)⁻¹ = (X^⊤WX)⁻¹

Where W is the block diagonal matrix of inverse of covariance matrix of ϵ.

Empirical Covariance

Empirical covariance, also known as the robust sandwich estimator, or “CR0”, is derived by replacing the covariance matrix of ϵ by observed covariance matrix.

cov(β̂) = (X^⊤WX)⁻¹(∑_iX_i^⊤W_iϵ̂_iϵ̂_i^⊤W_iX_i)(X^⊤WX)⁻¹ = (X^⊤WX)⁻¹(∑_iX_i^⊤L_iL_i^⊤ϵ̂_iϵ̂_i^⊤L_iL_i^⊤X_i)(X^⊤WX)⁻¹

Where W_i is the block diagonal part for subject i of W matrix, ϵ̂_i is the observed residuals for subject i, L_i is the Cholesky factor of Σ_i⁻¹ (W_i = L_iL_i^⊤).

See the detailed explanation of these formulas in the Weighted Least Square Empirical Covariance vignette.

Jackknife Covariance

Jackknife method in mmrm is the “leave-one-cluster-out” method. It is also known as “CR3”. Following McCaffrey and Bell (2003), we have

cov(β̂) = (X^⊤WX)⁻¹(∑_iX_i^⊤L_i(I_i − H_ii)⁻¹L_i^⊤ϵ̂_iϵ̂_i^⊤L_i(I_i − H_ii)⁻¹L_i^⊤X_i)(X^⊤WX)⁻¹

where

H_ii = X_i(X^⊤X)⁻¹X_i^⊤

Please note that in the paper there is an additional scale parameter $\frac{n-1}{n}$ where n is the number of subjects, here we do not include this parameter.

Bias-Reduced Covariance

Bias-reduced method, also known as “CR2”, provides unbiased under correct working model. Following McCaffrey and Bell (2003), we have cov(β̂) = (X^⊤WX)⁻¹(∑_iX_i^⊤L_i(I_i − H_ii)^−1/2L_i^⊤ϵ̂_iϵ̂_i^⊤L_i(I_i − H_ii)⁻¹L_i^⊤X_i)(X^⊤WX)⁻¹

where

H_ii = X_i(X^⊤X)⁻¹X_i^⊤

Kenward-Roger Covariance

Kenward-Roger covariance is an adjusted covariance matrix for small sample size. Details can be found in Kenward-Roger