Measurement Error in Nutritional Epidemiology

2.1. Basic concepts of measurement error

In epidemiology, errors in measurement of dietary intakes are widespread, and statistical methods for dealing with them have been developed in some depth.

The errors in assessing dietary intake depend upon the dietary instrument used. Commonly used instruments may be classified into three groups:

Longer-term self-report, including food frequency questionnaires. A food frequency questionnaire asks respondents to report their usual frequency of consumption of each food in a list of foods over a specific period of time, often 3 months, 6 months, or 1 year.
Shorter-term self-report, including 24-hour recalls and food records (sometimes called food diaries). A 24-hour recall asks the respondent to remember and report all foods and beverages consumed in the preceding 24 hours or during the preceding day; for a food record, the respondent records (in real time) the types and amounts of all foods and beverages consumed over one or more days.
Biomarkers, including recovery biomarkers, predictive biomarkers, and concentration biomarkers.

Measurement error in self-report instruments

The measurement error model that has been found most appropriate for most self-report dietary data is a specific version of the linear measurement error model in which the intercept is a random effect that varies across individuals (Kipnis et al, 2003). To make it clear that the intercept is a random effect, the model is often written as

$$X^*_i = \alpha_0 + a_{0i} + \alpha_X X_i + e_i$$

where the subscript $i$ denotes an individual, and the intercept has been split into a fixed part $\alpha_0 \text{,}$ (the average value of the parameter at the population level) and a random part $a_{0i}\text{,}$ (the value of the parameter for a particular individual in that population minus the population average). This random part has mean zero, and has been termed the person-specific bias. Thus, a person’s self-reported daily intake is a sum of the following terms:

A fixed intercept ($\alpha_0$)
a person-specific bias ($a_{0i}$)
a slope factor times the true usual intake ($\alpha_X X_i$ )
a random error ($e_i$)

If the intercept were zero, the person-specific bias zero, and the slope factor equal to one for all individuals (thereby satisfying the classical measurement error model), then the instrument would be unbiased at the individual level. Evidence has accumulated that neither 24-hour recalls nor multiple-day food records are unbiased instruments (Kipnis et al, 2003; Prentice et al, 2011; Prentice et al, 2013; Freedman et al, 2014). In most studies that have checked dietary measurement error, 24-hour recalls and multiple-day food records have come closer to unbiasedness than food frequency questionnaires. However, no known self-report instrument is truly unbiased. Usually the intercept is greater than zero and the slope factor is less than one, leading to the flattened slope phenomenon in which those who truly eat little tend to over-report their intake, while those who eat a lot tend to under-report their intake.

Much of our current knowledge regarding the measurement error characteristics of different self-report instruments comes from validation studies with recovery biomarkers as the unbiased reference instruments. The first such study that included several hundred participants was the Observing Protein and Energy (OPEN) study (Subar et al, 2003). This study documented the substantial under-reporting of energy and protein intakes that occurs using food frequency questionnaires and to a lesser extent 24-hour recalls. It also highlighted the low correlations seen between self-reported intakes of energy and protein with the true usual intake of these components, and the improved performance of food frequency questionnaires after energy adjustment of protein intake (Kipnis et al, 2003).

Following OPEN, several other large validation studies with recovery biomarkers have been conducted. Data from additional studies were pooled with those of the OPEN study in the Validation Studies Pooling Project and papers reporting the results of these studies pertaining to energy, protein, potassium, sodium and their densities have been published (Freedman et al, 2014; Freedman et al, 2015).

Measurement error in biomarkers

Recovery biomarkers are based upon recovery of specific biological products directly related to short-term dietary intake, but not subject to substantial inter-individual differences in metabolism. However, only a few are known: doubly-labeled water for energy intake (under the assumption that the person is in energy balance), 24-hour urinary nitrogen for protein intake, 24-hour urinary potassium for potassium intake, and 24-hour urinary sodium for sodium intake. The measurement error model most appropriate for recovery biomarker data is the classical measurement error model and these biomarkers are regarded as unbiased measures at the individual level. In addition their errors are thought to be independent of the errors in intakes obtained from self-report instruments.

Predictive biomarkers are dietary biomarkers that are characterized by a stable measurement error structure that allows them to be calibrated to true intake using data from feeding studies and used to predict true intake in an approximately unbiased manner. Good examples of such biomarkers are urinary sucrose and fructose for sugars intake (Tasevska et al, 2011). Serum lutein/zeaxanthin and certain other serum carotenoids may also qualify as a predictive biomarker for their respective carotenoid intakes (Freedman et al 2010c, Lampe et al 2017).

Concentration biomarkers are all dietary biomarkers that are correlated with dietary intake but are not recovery or predictive biomarkers. The measurement error model that is often used for them is the same linear measurement error model used for self-report data, although this may be at best a rough approximation to the true relationship.

The relationship between concentration biomarkers and true usual intake is often difficult to establish. The concentration biomarker may be affected not only by the average intake of the nutrient which it is intended to measure, but also by the time course of the intake of that nutrient, by the intakes of other nutrients, by physiological factors related to personal characteristics such as gender or race, and by other lifestyle factors such as physical activity or smoking. All of these render the detailed modeling of the biomarker-intake relationship extremely challenging (Kaaks and Ferrari, 2006). Sometimes, judicious use of the information that is available from previously conducted feeding studies can provide calibration equations for intake-based on biomarker levels (Freedman et al, 2011b), although the biomarkers for which this can be done most successfully are often termed predictive biomarkers.

Choice of reference measure in dietary validation studies

In order to adjust estimates for dietary measurement error, one needs estimates of parameters deriving from the measurement error model, and usually such estimates come from dietary validation studies in which individuals report their diet using the main study instrument and also complete an unbiased measurement that serves as the reference. Since no truly unbiased measurement is available for most dietary components, a common practice has been to use as the reference instrument a shorter-term self-report instrument that is more detailed and thought to be less biased. When the study instrument is a food frequency questionnaire, the reference instrument in the validation study has often been a food record or multiple 24-hour recalls.

Given that these shorter-term self-report instruments do not really meet the requirements of a reference instrument, the question arises as to whether there is any benefit from conducting validation studies based on their use. Some evidence suggests that validation studies using 24-hour recalls as reference instruments are useful for adjusting relative risk estimates based on food frequency questionnaire-reported intakes in models with continuous multiple dietary factors, for example, models that include energy adjustment (Freedman et al (2011a), Freedman et al (2017)). However, they may produce substantial bias in estimated correlations between FFQ-reported intakes and true usual intakes and should be interpreted with caution. In addition, since measurement error adjustment of relative risks between categories of intake depends on these correlation coefficients (Kipnis and Izmirlian, 2002), the same caution should be exercised when attempting to use 24-hour recalls to adjust relative risk estimates between categories of dietary intakes.

2.2. NCI models for short-term reference instruments

Recall from Section 1 of this primer that exposure measurements on an individual are rarely collected over an extended period of time, and the finite number of short-term exposure measurements that are available must be used to estimate the average, or usual exposure. Even when short-term exposure measurements are exact, the average of a few such measurements must still be regarded as an error-prone measurement of the usual exposure.

For most nutritional epidemiology studies, the targeted dietary measure of interest is usual intake. This is defined as the average long-term intake, although the period being referred to is often left unspecified. The exception is in surveys to monitor the diet of a population, where it is often described as the average intake over a specific year.

A measurement error model is needed to describe the relationship between an error prone intake measure and underlying, unobserved true usual intake. For shorter-term instruments, there are important and distinct modeling considerations for regularly consumed, episodically consumed, and never consumed dietary components. Regularly consumed dietary components give rise to continuous data, and may be described by the linear measurement error model as described above. However, episodically consumed and never consumed dietary components give rise to semi-continuous data, with a large proportion of zero amounts as well as a continuum of positive values. These data require a special measurement error model that accounts for periods without consumption of a dietary component.

The univariate NCI model is a two-part model for specifying usual reference intake of a single dietary component using two or more administrations of a shorter-term reference instrument. The first part of the model specifies the probability of reference consumption of a dietary component over a period (e.g., a day), and the second part describes the reference amount consumed over consumption periods. The usual reference usual intake is then the product of probability and amount. The corresponding measurement error model is based on the assumption that thereby specified reference usual intake is unbiased for true usual intake at the individual level (Tooze et al 2006, Kipnis et al 2009).

NOTE: In the absence of unbiased biomarkers for most food and nutrients, this model is commonly used with 24-hour recalls assumed to be an unbiased instrument. However, as stated previously, a 24-hour recall is not truly an unbiased instrument, and therefore this application of the model may not fully adjust for measurement error.

In addition to accounting for periods without consumption in estimating consumption amounts, strengths of the NCI model include allowing correlation between probability to consume and consumption amount, ability to distinguish within person variability from between person variability, and ability to relate other covariates to usual intake.

In more detail: The univariate NCI model for episodically consumed foods and nutrients

NOTE: The statistical notation used in Section 1 of this primer is different from the notation used below. In each Section of the primer, the symbols follow those typically used in the literature, and these have differed between the general statistical literature and the dietary measurement error literature.

Defining usual intake

The NCI model includes two features of usual intake.

The first is the individual probability to consume a dietary component in any given period (e.g., a day), $p_i = P(T_{ij} > 0 | i) \text{,}$ where subscript \(i$ refers to an individual, and $T_{ij}$ is the individual’s true intake of the food or nutrient in period $j\text{.}$

The second is the usual intake amount during a consumption period, $A_i = E(T_{ij} | i; T_{ij} > 0 ) \text{.}$ It follows that an individual’s usual intake, $T_i \text{,}$ is the product of the probability to consume in any given period and the average amount of intake during consumption periods, denoted by

$$T_i = E(T_{ij} | i ) = p_iA_i .$$

The measurement error model

We make two assumptions about the shorter-term dietary reference measure used:

The food or nutrient is reported as consumed in a certain period if and only if it was consumed in that period. If $R_{ij}$ denotes the measure of individual \(i$ on day $j\text{,}$ we write this assumption as \(P(R_{ij} > 0 | i) = P(T_{ij} > 0 | i) = p_i \text{.}$ It is also assumed that \(p_i$ is greater than zero for all $i\text{,}$ so that this version of the model does not formally include never-consumers of the food or nutrient, although $p_i$ may be arbitrarily small.
The shorter-term measure is unbiased for true usual intake on consumption days, and we write this assumption as $E(R_{ij} | i; R_{ij} > 0 ) = A_i \text{.}$

From this it follows that overall the shorter-term measure is unbiased for true usual intake at the individual level, that is

$$E(R_{ij} | i ) = p_iA_i = T_i .$$

In the first part of the NCI model, the consumption probability is modeled as a mixed effects logistic regression

$P(R_{ij} > 0 | i) = p_i = H(\beta_{10} + \beta^T_{X_1} X_{1ij} + u_{1i}), j=1, \dots, J_i$

where $H$ is the expit function (the inverse of the logistic), $\beta_{10}$ is an intercept, $X_{1ij}$ is a vector of covariates and $\beta_{X_1}$ is a vector of the coefficients of these covariates, $u_{1i} \sim \text{Normal}(0; \sigma^2_{u_1})$ is a random subject intercept term independent of $X_{1ij} \text{,}$ and $J_i$ is the number of days of report by individual $i\text{.}$ The random effect $u_{1i}$ allows an individual’s probability to deviate from the population level.

In the second part of the NCI model, the intake amounts reported during consumption periods are modeled. It is assumed a Box-Cox transformation of reported intake, $R_{ij}^{\#} \text{,}$ follows a mixed effects linear model

$(R_{ij}^{\#} | R_{ij} > 0) = \beta_{20} + \beta^T_{X_2} X_{2ij} + u_{2i} + \epsilon_{ij}, \; j = 1, \dots ,J_i$

where $X_{2ij}$ is a vector of covariates and $\beta_{X_2}$ a vector of their coefficients, $u_{2i} \sim \text{Normal}(0; \sigma^2_{u_2})$ is a random subject intercept term independent of $X_{1ij}$ and $X_{2ij}\text{,}$ and $\epsilon_{ij}$ is a random independent within-person variation term. The Box-Cox transformation parameters are chosen to make the distributions of person-specific random effects and the error term close to normal.

The two parts of the model are linked in two ways. First, the random effects, $u_{1i}$ and $u_{2i}\text{,}$ may be correlated, and second, both parts of the model may share common covariates among the components of $X_{1ij}$ and $X_{2ij}\text{,}$ also inducing correlation between probability of consumption and amount of consumption during consumption periods.

The univariate NCI model for episodically consumed dietary components handles a single dietary component only. However, it is often of interest to investigate two dietary components simultaneously, for example, in nutritional surveys when the food of interest and energy intake need to be assessed and some form of energy adjustment is desired. A bivariate extension of the NCI model for episodically-consumed components is available for such analyses. One of the intakes is allowed to be episodically-consumed, and the other must be regularly consumed. The model forms the basis for analyzing nutrient densities (Willet 2013: Chapter 11) and other ratios of dietary intakes. The model may be used to analyze these not only in the context of estimating distributions of usual intake, but also in relating nutrient densities or other ratios to health outcomes.

In more detail: The bivariate NCI model for episodically consumed foods and nutrients

There are two slightly different versions of the bivariate NCI model, one of them more general than the other. The first, less general version was used in analyses of usual intake distributions of ratios of intakes (Freedman et al, 2010a) and components of the Healthy Eating Index (Freedman et al, 2010b). The second, more general version was described by Kipnis et al (2016).

First, less general version of the bivariate NCI model

We denote the episodically consumed dietary component by (for food) and the regularly consumed component by (for energy). For individual $i, \; i = 1, \dots , n\text{,}$ let

$T_{Fi}$ = true usual intake of the episodically consumed food

$T_{Ei}$ = true usual intake of energy

$R_{Fij}$ = Reported intake of food in period $j, \; j = 1, \dots , J_i$

$R_{Eij}$ = Reported intake of energy in period $j, \; j = 1, \dots , J_i$

$X_{ij}$ = vector of covariates relevant to period $j, \; j = 1, \dots , J_i$

As with the univariate NCI model, we assume that the shorter-term measure is unbiased on the original scale,

$E(R_{Fij}) = T_{Fi}$

$E(R_{Eij}) = T_{Ei}.$

We also assume, as an extension of the univariate NCI model, the following three-part model. The first two parts follow the basic univariate NCI model,

$$P(R_{Fij} > 0 | i) = p_i = H( \beta_{10} + \beta^T_{X_1} X_{1ij} + u_{1i} ), \; j = 1, \dots ,J_i$$

$$(R_{Fij}^{\#} | R_{Fij} > 0) = \beta_{20} + \beta^T_{X_2} X_{2ij} + u_{2i} + \epsilon_{2ij}, \; j = 1, \dots ,J_i,$$

where $R_{Fij}^{\#}$ is a Box-Cox transformed value of $R_{Fij} \text{.}$

The third part of the bivariate NCI model describes the model for energy intake

$$R_{Eij}^{\#} = \beta_{30} + \beta^T_{X_3} X_{3ij} + u_{3i} + \epsilon_{3ij}, \; j = 1, \dots ,J_i,$$

where $R_{Eij}^{\#}$ is a Box-Cox transformed value of $R_{Eij} \text{.}$

The terms $(u_{1i}, u_{2i}, u_{3i})$ are random effects that have a joint normal distribution with mean zero and an unstructured covariance matrix, and the terms $(\epsilon_{2ij}, \epsilon_{3ij})$ are within-person random errors that have a joint normal distribution with mean zero, variances $\sigma^2_{\epsilon_2}, \; \sigma^2_{\epsilon_3}$ and correlation $\rho_{23}\text{.}$ The terms $(\epsilon_{2ij}, \epsilon_{3ij})$ are independent of $(u_{1i}, u_{2i}, u_{3i})\text{.}$ In addition, values of $(\epsilon_{2ij}, \epsilon _{3ij})$ are independent across repeats. The terms $\beta_{10}, \beta_{20}$ and $\beta_{30}$ are scalars. The $X$ ’s are vectors of covariates and do not need to include the same covariates for each part of the model. The terms $\beta_{X_1}, \beta_{X_2}$ and $\beta_{X_3}$ are also vectors, with the same number of elements as the corresponding $X\text{.}$

The Box-Cox transformation parameters are chosen to make the distributions of the variables close to normal. For the bivariate model, these parameters are sometimes estimated prior to the estimation of the parameters in the three-part model.

Second, more general version of the bivariate NCI model

In the second version of the NCI bivariate model the probability of consuming the food in a given period follows a probit rather than a logistic model. This change was made to simplify the specification of a second modification, namely that the energy intake in a given period can depend on whether the food is consumed in that period.

Let $I_{Fij} = I(R_{Fij} > 0), \; j = 1, \dots ,J_i \text{,}$ where $I(x)$ is the indicator function.

We assume that this binary indicator $I_{Fij}$ results from dichotomizing a continuous latent variable $R_{F1ij} \text{,}$ written as

$I_{Fij} = I(R_{F1ij} > 0), \; j = 1, \dots ,J_i \text{,}$ where

$$R_{F1ij} = \beta_{10} + \beta^T_{X_1} X_{1ij} + u_{1i} + \epsilon_{1ij}, \; j = 1, \dots ,J_i ,$$ with $u_{1i} \sim \text{Normal}(0; \sigma^2_{u_1})$ and $\epsilon_{1ij} \sim \text{Normal}(0; \sigma^2_{\epsilon_1})$ independent of each other and of $X_{1ij}\text{.}$ For identifiability, $\sigma^2_{\epsilon_1}$ has to be fixed, and without loss of generality we set $\sigma^2_{\epsilon_1} = 1\text{.}$ Note that this model is equivalent to specifying the probability of consumption on any given day using a mixed effects probit regression. The advantage of this specification is that it allows the latent variable $\epsilon_{1ij}$ to be correlated with its counterpart in the model for energy intake.

The second and third parts of the model are the same as in the first version of the NCI bivariate model, namely:

$$(R_{F2ij}^{\#} | R_{Fij} > 0) = \beta_{20} + \beta^T_{X_2} X_{2ij} + u_{2i} + \epsilon_{2ij}, \; j = 1, \dots ,J_i,$$

where $R_{F2ij}^{\#}$ is a Box-Cox transformed value of $R_{Fij} \text{,}$ and

$$R_{Eij}^{\#} = \beta_{30} + \beta^T_{X_3} X_{3ij} + u_{3i} + \epsilon_{3ij}, \; j = 1, \dots ,J_i,$$

where $R_{Eij}^{\#}$ is a Box-Cox transformed value of $R_{Eij} \text{.}$

As in the first version, the random effects $(u_{1i}, u_{2i}, u_{3i})$ are allowed to be mutually correlated. The within-person errors $(\epsilon_{1ij}, \epsilon_{2ij}, \epsilon_{3ij})$ are independent of $(u_{1i}, u_{2i}, u_{3i})$ and there are no across-time correlations. The energy within-person error, $\epsilon_{3ij}\text{,}$ is allowed to be correlated with its counterparts $\epsilon_{1ij}$ and $\epsilon_{2ij}\text{.}$ However, $\epsilon_{1ij}$ and $\epsilon_{2ij}$ are assumed uncorrelated so that marginally $R_{Fij}$ and $R_{Eij}$ follow the NCI univariate model for episodically-consumed and regularly-consumed dietary components, respectively. Note that although the above models are written for a single episodically consumed dietary component together with a single regularly-consumed component, they can also accommodate two regularly-consumed components, such as saturated and total dietary fat intakes (and hence also their ratio).

There are also occasions when it is of interest to investigate several dietary components simultaneously, for example, when estimating the population distribution of the total Healthy Eating Index score based on usual intake or relating this score to a health outcome. A multivariate extension of the NCI model for episodically-consumed components is available for such analyses.

In more detail: The multivariate NCI model for episodically consumed foods and nutrients

The multivariate NCI model is a natural extension of the second version of the bivariate NCI model (Zhang et al 2011b). Suppose that there are $r$ episodically consumed and $s$ regularly consumed foods or nutrients that are to be jointly analyzed. Label the episodically consumed components $Fk, \; k=1, \dots ,r,$ and the regularly consumed components $Et, \; t=1, \dots ,s\text{.}$ We form two variables for each replicate measurement of $Fk, \; \; R_{Fk1ij}$ that serves as an indicator of whether the food has been consumed by person $i$ in period $j\text{,}$ and $R_{Fk2ij}^{\#}$ that is a Box-Cox transformation of the amount reported by person $i$ in period $j$ , given he/she consumed. Also, we form $R_{Etij}^{\#}$ a Box-Cox transformed amount consumed for each replicate measurement of component $Et\text{.}$

Thus the complete model is as follows:

$$R_{Fk1ij} = \beta_{Fk10} + \beta^T_{FkX1} X_{F1ij} + u_{Fk1i} + \epsilon_{Fk1ij}, \; j = 1, \dots ,J_i; \; k=1, \dots ,r$$

$$(R_{Fk2ij}^{\#} | R_{Fk1ij} > 0) = \beta_{Fk20} + \beta^T_{FkX2} X_{F2ij} + u_{Fk2i} + \epsilon_{Fk2ij}, \; j = 1, \dots ,J_i; \; k=1, \dots ,r$$

$$R_{Etij}^{\#} = \beta_{Et0} + \beta^T_{EtX} X_{Eij} + u_{Eti} + \epsilon_{Etij}, \; j = 1, \dots ,J_i; \; t=1, \dots ,s.$$

The $u$ terms are random subject-specific intercepts that have a multivariate normal distribution with an unstructured covariance matrix. The $\epsilon$ terms also have a multivariate normal distribution that is independent of the $u$ ’s and $X$ ’s. The covariance matrix of the epsilons has the same type of restrictions that were imposed in the second version of the bivariate NCI model, i.e.

$$\text{var} (\epsilon_{Fk1ij})=1, \; j = 1, \dots ,J_i; \; k=1, \dots ,r \text { and cov} (\epsilon_{Fk1ij}, \epsilon_{Fk2ij})=0, \; j = 1, \dots ,J_i; \; k=1, \dots ,r.$$

In the NCI model for episodically consumed foods and nutrients, it is assumed that all individuals consume the dietary component occasionally, even if rarely and in very small amounts. However, there are some foods or nutrients never consumed by a substantial proportion of the population, for example, fish or alcohol. For these components, one version of the NCI model allows estimation of the proportion of never-consumers, even in bivariate and multivariate problems, but only when one of the dietary components has never-consumers. The statistical problem that arises is that it becomes necessary to distinguish between reports of zero intake that come from never-consumers and those that come from episodic consumers. This is usually very difficult to do without either (a) a considerable number of repeat measurements per person or (b) a covariate that has a strong correlation with being a never-consumer. It is not recommended to use this model unless either condition (a) or (b) pertains, and it is preferable that both conditions apply.

In more detail: The NCI model for episodically consumed foods and nutrients with never consumers

The extension of the NCI model to include never-consumers is achieved as follows. A latent binary variable $N_i$ $=1$ for a never-consumer and $0$ for a consumer) is introduced with

$$P(N_i = 1 ) = \Phi( \alpha_0 + \alpha^T_G G_i),$$

where $\Phi$ is the standard normal cumulative probability distribution function.

The vector of covariates $G_i$ includes those covariates that are expected to be associated with being a never-consumer. This forms an extra part of the model together with the two parts included in the basic NCI model. However, to make the likelihood easier to analyze, a small modification to the basic NCI model is also made. In this version we write the probability of consumption among consumers as:

$$P(R_{ij} > 0 | i ) = p_i = \Phi( \beta_{10} + \beta^T_{X_1} X_{1ij} + u_{1i}), \; j=1, \dots, J_i$$

because $\Phi$ is a good approximation to the logistic function over most of the range, this modification has little or no impact on the estimates of the model parameters. One more difference between this later formulation of the NCI model and the original formulation is that the later formulation accommodates correlations between the consumption of one episodically-consumed food and the amount of another food or nutrient eaten in the same period. For more details of the never-consumers model and how it is fit, see Bhadra et al (2016).

2.3. Impact of dietary measurement error on nutrition studies

Nutritional epidemiology studies vary greatly in their aims and designs. Below is a description of how non-differential dietary measurement error can impact results from three classes of study: etiologic studies, surveillance studies, and dietary interventions.

Impact on studies evaluating the association of diet with an outcome when diet is measured with error

Etiologic studies are conducted to evaluate associations between dietary intakes and health outcomes. Most commonly, food frequency questionnaires have been used as the self-report instrument in prospective cohort studies, but some studies also administer a series of shorter-term assessments, such as 24-hour recalls or food records (Bingham et al, 2008). One or more shorter-term self-report instruments have been used in cross-sectional studies.

The target estimate is typically an association in the form of a relative risk or hazard ratio obtained as the exponent of a regression coefficient for the dietary component of interest. The outcome model typically includes other explanatory variables that are potential confounders.

It is quite common to use an energy-adjusted value for the dietary component of interest and include self-reported total energy intake as an extra explanatory variable.

The energy-adjusted intake is obtained by first performing a linear regression of the nutrient intake on energy intake, then calculating the residual for the observation in question, and finally adding the group mean nutrient intake.

One of the main reasons for using energy adjustment is that, for food frequency questionnaire reports, it has been found that the correlations of reported nutrient densities and energy-adjusted intakes with true values are much higher than for unadjusted intakes (Freedman et al 2014, Freedman et al, 2015). Thus, using energy-adjustment reduces the impact of dietary measurement error on estimates of diet-health outcome associations.

The impact of the dietary measurement error when diet is the exposure in etiologic studies is to (a) bias the estimate of the relative risk, and (b) reduce the power to detect an association.

The direction of the bias of the estimated relative risk is usually towards the null value of 1, but in rare cases it is possible for measurement error to cause exaggeration (Freedman et al 2011a, Freedman et al, 2014, Freedman et al, 2015).

Impact on studies evaluating the population distribution of diet when diet is measured with error

Dietary surveys are conducted to obtain an understanding of the dietary intake of a population. In the US, the most well-known surveillance study is the National Health and Nutrition Examination Survey (NHANES). The instrument that is used in NHANES is the 24-hour recall. Each participant is asked to complete two 24-hour recalls, the repeated assessment made within a few weeks of the first. The main aim is to estimate the population distribution of usual daily intake for a range of dietary components.

If the self-report instrument were unbiased, conforming to the classical measurement error model, then the impact of dietary measurement error on the estimate of the population distribution would be to overestimate the spread of the distribution because the random within-person error gets wrongly incorporated into the estimate. Thus lower percentiles would be underestimated and upper percentiles would be overestimated. In practice, if a 24-hour recall is the survey instrument, then for most dietary components there is also some systematic error involved that tends to induce over-reporting of lower levels of intake and under-reporting of higher levels of intake. The effects of the random and systematic error together typically lead to considerable underestimation of the lower percentiles, some underestimation of the median, and some overestimation of the upper percentiles. Freedman et al (2004) and Yanetz et al (2008) discuss the problem of estimating the population distribution when the measured intake has linear measurement error.

Impact on studies where diet is an outcome and measured with error

Data from dietary surveys may be used to investigate determinants of dietary intake, or an intervention or experimental study may be conducted to examine whether dietary intake may be changed.

If the study is to examine whether the intervention effects a change in dietary intake, then participants are likely to be asked to self-report their dietary intakes at baseline, during the intervention and at the end of the study, and self-reported diet is then used as the main outcome variable. Thus, the target estimate is often the difference in mean self-reported intakes of a specific dietary component in the intervention and control groups.

If the self-report instrument conforms to the linear measurement error model, then the association between the exposure and dietary intake will be biased. For the dietary components that are known about, the direction of this bias is toward the null when the measurement error is non-differential.

It has been documented in some intervention trials that dietary measurement error is differential, with the intervention participants tending to report intakes closer to the study target than what they are truly eating. This tendency will exaggerate the difference in dietary intakes between the two groups, and will bias the estimated treatment effect away from the null. This website does not provide statistical software for adjusting for the effects of measurement error in self-reports in dietary intervention studies. To read about the design and analysis of intervention trials that specify the response variable to be a dietary intake, see Keogh et al (2016).

2.4. NCI method of adjusting for dietary measurement error

Measurement error causes biases in target estimates and loss of precision in epidemiologic studies. The NCI method may be applied to reduce the bias caused by dietary measurement error. The NCI method is based on an assumption that a shorter-term dietary assessment is unbiased at the individual level for usual intake, and shorter-term intakes are linked to usual intake through the NCI model for measurement error.

There are many combinations of circumstances under which one may be investigating relationships between dietary intakes and a health outcome, depending on the dietary data available and the number and type of dietary components that are of interest. The NCI method provides options for a wide range of combinations of dietary components; the general statistical method used is common to all of these options. The dietary data that we typically deal with are of two types:

A food frequency questionnaire completed by all participants, plus 24-hour recalls in a subsample, with a substantial number in that subsample having at least one repeat 24-hour recall
24-hour recalls completed by all participants, with a substantial subsample having at least one repeat 24-hour recall (participants may or may not have an FFQ)

The NCI method may be implemented using software provided in Section 3 of this primer. This software allows analysis of regularly, episodically, or never consumed dietary components, absolute intakes or densities, and single or multiple dietary components.

The NCI method allows calculating standard errors of estimated parameters by either non-parametric bootstrap or balanced repeated replication (BRR). The choice of which method to use depends on the nature of the study and the nature of the estimate. When the data are from a complex sample survey and percentiles of the distribution are being estimated, then the BRR method is recommended. The bootstrap should be used for data from other designs, such as simple random surveys, convenience samples, and cohort studies.

Analyses may include one dietary variable, two dietary variables, or multiple dietary variables, but the fitting of the multivariate model is much more complex than in the univariate and bivariate analyses and requires the use of Markov Chain Monte Carlo technology.

Analysis of dietary components with a substantial proportion of never consumers also requires the use of Markov Chain Monte Carlo technology to fit the model. A food that has never-consumers may be analyzed in a multivariate model alongside other episodically- or regularly-consumed dietary components. However, only one food that has never-consumers can be included in the multivariate model.

For regularly-consumed foods, the measurement error model is a simplified version of that for episodically-consumed foods. Specifically, the first part of the NCI model, pertaining to the probability of consumption on a particular day, is dropped since it is now assumed that the probability equals one. The simplified model reverts to a single part for the amount consumed on a particular day.

Adjusting diet and outcome associations when diet is measured with error

When estimating a relative risk, hazard ratio, or other measure of association for a chosen health outcome, naively using each individual’s shorter-term dietary intake as their usual intake leads to a biased estimate of the association.

Estimates may be adjusted for measurement error using the NCI method, which is a regression calibration method. The inclusion of covariates in the NCI model allows for it to be applied as an enhanced regression calibration method, recovering some lost precision (Kipnis 2009). The NCI method adjusts for the random within-person error of shorter-term assessments. Such an adjustment is made possible by information provided from repeat administrations of the instrument.

NOTE: An assumption of the NCI method is the reference instrument is unbiased. Shorter-term dietary self-reports are often applied as reference instruments, but are not unbiased and therefore cannot be expected to totally eliminate the bias in estimated risk parameters that is caused by dietary measurement error.

Similar adjustment methods can be used for longitudinal and cross-sectional analysis of dietary intakes and health outcomes. If the study design involves drawing a sample from the population using a method that is not simple random sampling, then the analysis may require use of survey sampling weights. For a discussion of when to use sampling weights, see Korn and Graubard (1991).

In general, there are four main steps to implementing the NCI method to adjust estimates of association. Depending on the analysis at hand, there may be preliminary steps to the analysis.

In more detail: The NCI method for adjusting estimates of diet and outcome associations

Number of Dietary Components	1	2	>=3
NCI Model	Univariate	Bivariate	Multivariate
NCI Method Step 1	Fit NCI model parameters using the method of maximum likelihood using the procedure for nonlinear mixed regression models, NLMIXED, in the SAS package	Fit NCI model parameters using the method of maximum likelihood using the procedure for nonlinear mixed regression models, NLMIXED, in the SAS package	Fit NCI model parameters using a Markov Chain Monte Carlo method that computes the Bayesian posterior joint distribution of the model parameters after ascribing to them default non-informative prior distributions
NCI Method Step 2	Generate predicted intakes: Calculate the conditional expectation of the individual's true usual intake given the observed reported intakes and other covariates	Generate predicted intakes: Calculate the conditional expectation of the individual's true usual intake given the observed reported intakes and other covariates	Generate predicted intakes: Calculate the conditional expectation of the individual's true usual intake given the observed reported intakes and other covariates
NCI Method Step 3	Substitute predicted usual intakes into a regression model linking the outcome to dietary intakes and estimate the corresponding regression parameters	Substitute predicted usual intakes into a regression model linking the outcome to dietary intakes and estimate the corresponding regression parameters	Substitute predicted usual intakes into a regression model linking the outcome to dietary intakes and estimate the corresponding regression parameters
NCI Method Step 4	Repeat the above analysis steps using replicate datasets (bootstrap) or replicate weights (BRR) to obtain standard errors of estimated regression parameters	Repeat the above analysis steps using replicate datasets (bootstrap) or replicate weights (BRR) to obtain standard errors of estimated regression parameters	Repeat the above analysis steps using replicate datasets (bootstrap) or replicate weights (BRR) to obtain standard errors of estimated regression parameters

Adjusting population distributions of intake when diet is measured with error

When estimating the distribution of usual intake in a population, naively using each individual’s shorter-term dietary intake as their usual intake leads to a biased estimate of this distribution.

The NCI method adjusts the distribution for the random within-person error of the shorter-term assessments. Such an adjustment is made possible by information provided from repeat administrations of the instrument. The NCI method can handle using the sampling weights derived from complex random sample designs. The NCI method can also incorporate information from another dietary instrument, such as a food frequency questionnaire administered in addition to a 24-hour recall. This can be very helpful in estimating distributions of episodically-consumed dietary components.

In general, there are three main steps to implementing the NCI method to adjust distribution estimates. Depending on the analysis at hand, there may be preliminary steps to the analysis.

In more detail: The NCI method for adjusting population distributions of intake

Number of Dietary Components	1	2	>=3
NCI Model	Univariate	Bivariate	Multivariate
NCI Method Step 1	Fit NCI model parameters using the method of maximum likelihood using the procedure for nonlinear mixed regression models, NLMIXED, in the SAS package	Fit NCI model parameters using the method of maximum likelihood using the procedure for nonlinear mixed regression models, NLMIXED, in the SAS package	Fit NCI model parameters using a Markov Chain Monte Carlo method that computes the Bayesian posterior joint distribution of the model parameters
NCI MethodStep 2	Use a Monte Carlo technique to simulate a large set of usual intakes according to the estimated parameters Obtain summary statistics for the population using this sample of usual intakes	Use a Monte Carlo technique to simulate a large set of usual intakes according to the estimated parameters Obtain summary statistics for the population using this sample of usual intakes	Use a Monte Carlo technique to simulate a large set of usual intakes according to the estimated parameters Obtain summary statistics for the population using this sample of usual intakes
NCI Method Step 3	Repeat the above analysis steps using replicate weights (BRR) or replicate datasets (bootstrap) to obtain standard errors of the estimated summary statistics	Repeat the above analysis steps using replicate weights (BRR) or replicate datasets (bootstrap) to obtain standard errors of the estimated summary statistics	Repeat the above analysis steps using replicate weights (BRR) or replicate datasets (bootstrap) to obtain standard errors of the estimated summary statistics

The NCI method defaults to estimating the distribution of usual intake for the population represented by all data used in the estimation exercise. However, estimated distributions for subpopulations are often of interest. One could produce such estimates by stratification of the full sample. Alternatively, the NCI method allows the use of variables for subpopulation membership as covariates in a mixed effects model. The two approaches are not equivalent, because the stratification approach permits all model parameters to change within a subpopulation, while the covariate approach assumes homoscedastic variance components for the random effect and residual error terms across all subpopulations. In some cases, a combination of the stratification and covariate approaches may be desired.

For example, cutpoints for inadequate and excessive nutrient intake vary by sex, age, and pregnancy status. Even in a large scale national survey, there may be too few pregnant women to allow stable estimation using the stratification approach. On the other hand, there may be solid evidence that men, women, and children should be analyzed as three separate groups. In this case, it might be reasonable to split the sample into children of both sexes, adult males, and adult females, and run three analyses where the models include covariates for age groups, the model for women includes a covariate for pregnancy status, and the model for children includes a covariate for sex. Results from the three separate runs can be combined to estimate, e.g., the distribution of the entire national population, or the distribution of all adults. Distributions for subpopulations wholly contained within one of the stratified subsamples (e.g., pregnant women) can be estimated using the results of a single run.

There are other methods available to estimate usual intake distributions.

The Iowa State University (ISU) method (Nusser et al (1996)) is able to deal with single regularly-consumed foods and nutrients. It deals also with episodically-consumed foods and nutrients assuming that the probability to consume on a given day and the amount consumed on that day are independent. It does not handle covariates related to intake, foods with never-consumers, nor bivariate or multivariate distributions. The ISU method may be implemented using the package PC-SIDE (PC Software for Intake Distribution Estimation).

The Multiple Source Method (Harttig et al (2011), Haubrock et al (2011)) handles episodically-consumed foods and foods with never-consumers, but the statistical methods used in the implementation are not the same as for the NCI method. The Multiple Source Method does not deal with bivariate or multivariate distributions.

The Statistical Program to Assess Dietary Exposure (SPADE) method (Dekkers et al (2014)) handles episodically-consumed foods and foods with never-consumers, but the statistical methods used in the implementation are not the same as for the NCI method. The SPADE method also analyzes several dietary components simultaneously.

Contact the Biometry Research Group

For Regular Mail

For Courier Packages