Effect of Refusal Avoidance Training on Unit Nonresponse Rates
Table of Contents:
In their examination of survey participation, Groves and McGonagle (2001,250-251) assert that two interviewer strategies - tailoring behavior to the perceived features of the sample person and maintaining interaction with the sample person - play a crucial role in gaining the cooperation of potential respondents. They posit that "maintaining interaction is the essential condition of tailoring, for the longer the conversation is in progress, the more cues the interviewer will be able to obtain from the householder" (p. 251). Moreover, they argue that the longer the interaction lasts, the harder it is for the sample unit to refuse to participate. Thus, the first research question addressed concerns whether refusal avoidance training improves response rates (see Q1 in Table 4.1).
Effect of Interviewer Training on Data Quality
Especially in the case of measurement error, interviewers can be trained to avoid certain verbal or non-verbal behaviors that may influence respondents. Interviewer effects reflect the tendency that responses collected by one interviewer are more similar to each other than responses from different interviewers (Groves and Magilavy 1986). Reasons for interviewer effects include the activation of social norms by the interviewer's presence (Bosnjak 2017; Miller and Cannell 1982) and systematic errors in administering the survey (e.g., failure to read questions as worded). A typical example is the tendency of white respondents to report more liberal responses to a black interviewer for racial topics (e.g., Schaeffer 1980). Because interviewer training alerts interviewers to the importance of standardized interviewing with the aim of preventing, or minimizing, interviewer effects on survey responses, our second question examines whether measurement error is reduced if interviewers undergo specific questionnaire administration training (see Q2 in Table 4.1).
Effect Size Heterogeneity
Unfortunately, interviewer training is not standardized or homogeneous in terms of duration, content, and training procedures, although initial efforts have been made (AAPOR
Research Questions Addressed by the Meta-analysis and Systematic Analysis
2016; Daikeler et al. 2017; ISO 2012; Viterna and Maynard 2002). Our third research question explores whether the effect sizes are heterogeneous (see Q3 in Table 4.1). Because of the lack of standardization, heterogeneous training outcomes, and thus effect size heterogeneity, can be expected. Heterogeneous distributions of effect size would imply that the success of interviewer training depends more on the content and methods of training or other factors such as the population and study content than on the training itself; alternatively, there may be other variables that confound these effects that cannot be disentangled in these analyses. Accordingly, a number of other training features must be examined more closely in order to be able to make statements on what constitutes successful training.
Training Features That May Improve Data Quality
Interviewer training duration. Learning theory suggests that learning progress typically follows an S-shaped curve, starting slowly, accelerating, and then leveling off (Thorndike 1913). If the learning curve flattens out or becomes horizontal, learning progress stagnates. This phenomenon, which is referred to as a learning plateau (Thorndike 1913, 99), occurs during the acquisition of complex skills such as learning the behaviors and techniques for standardized interviewing. Our research attempts to evaluate training durations that enable interviewers to learn the skills they need to avoid refusals and reduce interviewer effects (see Q4 in Table 4.1).
Interviewer training methods and determinants of effectiveness. According to Knowles, Holton, and Swanson's (2005) adult learning theory (see review in Tusting and Barton 2003), one reason why adults learn differently than children is that they can already draw on a variety of experiences which affect their actions. Hence, learning techniques that incorporate and extend the experiences of the learners are the most effective, e.g., an interviewer might remember that rephrasing a question in the past led to better understanding of the respondent, and in the training rephrasing techniques could then be taught to extend this skill. Furthermore, individuals differ in their preferred learning style; some react more to visual information, some to auditory, and others to kinesthetic information (Kelly 2010). Knowles, Holton, and Swanson (2005) posited that adults primarily would like to learn new skills because they are confronted with a concrete problem in their everyday life; for example, the concrete problem for the interviewer is to handle the question-answer process as correctly and standardized as possible, so they are willing to learn the necessary skills. According to Knowles, Holton, and Swanson (2005) and Tusting and Barton (2003), most adults prefer self-directed, problem-centered learning.
Against this background, a flexible blended-learning approach to adult learning, which combines traditional face-to-face instruction with online learning, seems especially promising (Means et al. 2013). Blended learning combines the advantages of online learning, such as flexibility in terms of time and place, with those of face-to-face instruction, such as direct interaction with trainers and other trainees and live feedback. Table 4.1 (Q5) summarizes this as our fifth research question.