Desktop version

Home arrow Mathematics

  • Increase font
  • Decrease font

<<   CONTENTS   >>

The Basic Model

A frequently used metric for evaluating interviewer variance is the intra-class correlation coefficient (ICC), also referred to as the intra-interviewer correlation (IIC) in this context because the classes are defined by interviewers (Hox 1994). This coefficient expresses the homogeneity of the obtained answers within interviewers, or equivalently the ratio of the between-interviewer variance in a variable of interest to the total variance in that variable. To calculate the within-interviewer and the between-interviewer variance components that are critical for computing the IIC, it is necessary to take into account the two- level hierarchical data structure in which respondents are nested within interviewers. A two-level random intercept model with У as the dependent variable and no independent variables (i.e., a null model) is generally the starting point to estimate the within- and between-interviewer variance components.

The standard expression of this model is

In this model, Y); is the value of the dependent variable Y for respondent i (i = 1, ..., N) interviewed by interviewer j (j = 1,..., /), у is the fixed (overall) intercept, m0; is the inter- viewer-specific part of the random intercept for interviewer j (interviewer level), and is the residual error term for respondent i interviewed by interviewer j (respondent level),

with e,i ~ N(0,f2) and u0j ~ N(0,<72).

There are significant differences between interviewers when al differs significantly from zero. The IIC for each variable Y is estimated as the proportion of the total variance in the dependent variable Y that is explained by the differences between interviewers:

This quantitative expression of the interviewer effect and interpretation of the IIC is correct under the "comparable respondent groups" assumption. This means that the differences between interviewers are not arising because interviewers did not interview comparable groups of respondents. To control for possible differences between the interviewers in terms of the groups of respondents interviewed, one can expand the basic model in Equation 22.1 by including relevant background characteristics of respondents as predictors of the variable Y. Accordingly, the interviewer effects express the variability between interviewers after adjusting for these respondent characteristics. Notice that the interpretation of the relationships of the independent variables with the dependent variable in the expanded basic model concerns explaining the variability in the dependent variable, and not explaining the differences between interviewers. In fact, the relationships between respondent characteristics and interviewer effects are not specified in the model.

In addition to the elaboration of the basic model with respondent characteristics, one could also expand the model by including interviewer characteristics (e.g., experience, workload, gender, etc.). This further extension of the basic model allows us to explain the observed differences between interviewers. If these characteristics partly explain the interviewer variance, they provide some insights into the mechanisms underlying interviewer effects. This elaboration of the basic model to include respondent and interviewer characteristics results in the following model:

with Xri. indicating respondent characteristics X,. (r = 1,..., R) measured for respondent i by interviewer j, and W„ indicating interviewer characteristics W, (f = 1,..., T) measured for interviewer j. Notice that the inclusion of respondent characteristics (the "control by modeling" approach) is only a partial solution to the fact that in most cases there is no interpenetrated sample assignment to the interviewers (the "control by design" approach).

The extended basic model with respondent and interviewer characteristics (Equation 22.2) allows analysts to assess the relationships between the interviewer- specific means and interviewer characteristics, but not the relationships between interviewer effects and respondent characteristics. However, it is reasonable to assume that some groups of respondents have more difficulties in understanding and answering the questions, which intensifies the interaction between interviewer and respondent, increasing the risk of interviewer effects. This can result in higher IICs in these respondent groups. The specification of the basic multilevel model does not allow us to investigate whether the occurrence of interviewer effects varies across different respondent subgroups.

In this chapter, we focus on two procedures based on the basic model that allow for a direct investigation of the relationships between respondent characteristics and the occurrence of interviewer effects (as measured by the IICs). We do not discuss the ability of interviewer characteristics to explain interviewer effects. We use the respondent's education level to illustrate the two procedures, examining whether interviewer effects differ according to the respondent's education level. In the next section, we present the data used in the analysis and the results of a preliminary analysis.

<<   CONTENTS   >>

Related topics