What Do Interviewers Learn?: Changes in Interview Length and Interviewer Behaviors over the Field Period
Table of Contents:
Interviewers are important actors in telephone surveys. By setting the pace for an interview, interviewers communicate the amount of time and cognitive effort respondents should put into their task. It is well-established that interviewers vary widely in the time they spend administering a survey, and that this time changes over the course of the data collection period as interviewers gain experience (Bohme and Stohr 2014; Kirchner and Olson 2017; Loosveldt and Beullens 2013a, 2013b; Olson and Bilgen 2011; Olson and Peytchev 2007). In particular, interviewers get faster as they gain experience over the field period of a survey.
The within-survey effect of experience on interview length is generally attributed to interviewer learning effects. In particular, a learning effect occurs when interviewers learn how to change their behaviors to more quickly administer questions. This can include positive changes in behaviors over the field period such as error-free administration of questions or negative changes such as shortening questions (i.e., non-standardization) or avoiding positive, time-consuming behaviors like probing or verifying answers (e.g., Bohme and Stohr 2014; Kirchner and Olson 2017; Loosveldt and Beullens 2013a, 2013b; Olson and Peytchev 2007). Other hypotheses about why the length of interview changes over the course of the data collection period, including characteristics of the respondents or interviewers or differential respondent motivation correlated with their response propensity, have not explained away the learning effect (e.g., Kirchner and Olson 2017). However, Kirchner and Olson (2017) found that a measure of the interaction between interviewers and respondents - the number of words spoken by the interviewer and by the respondent - partially mediated the interviewer learning effect.
Despite the well-replicated finding that interviewers speed up over the field period, what behaviors change and whether they explain the decrease in interview length over the course of data collection has not been previously examined in published articles. This chapter examines two research questions:
RQ1: What standardized, nonstandardized, and inefficient interviewer behaviors change over the course of the data collection period?
RQ2: Do these behaviors account for changes in interview length over the course of the data collection period?
To answer these questions, we draw on two nationally representative US telephone surveys of adults. Both surveys were audio-recorded and transcribed. Interviewer and respondent behaviors were coded at the conversational turn level, allowing a detailed examination of the changes in interviewer behaviors over the course of the field period. We focus on interviewer behaviors, as the learning hypothesis focuses primarily on changes by the interviewer, although interviewer behaviors inevitably affect respondent behaviors as well.
Hypotheses for Behaviors Affected by Interviewer Learning
There are three main hypotheses about what interviewers may "learn" as they conduct interviews over the course of the field period. First, interviewers may learn to omit or shorten certain standardized interviewer behaviors (i.e., "good" behaviors). Standardized behaviors include reading questions exactly as worded, using nondirective probes, repeating the respondents' answers to verify what they said, clarifying the question wording, and providing appropriate feedback to the respondent (Fowler and Mangione 1990). The standardized "good" behaviors may be eliminated as interviewers learn what may be shortcut, become bored or frustrated with certain questions, think that certain questions are emotionally draining for follow-up, or think that they remember the question wording, and thus do not read the item on the questionnaire directly (Kaplan and Yu Chapter 5; Ongena and Dijkstra 2007). As the field period progresses and interviewers learn from previous respondents' answers, they also may be more likely to enter a response that is not directly codable rather than probe nondirectively for a codable response (Ongena and Dijkstra
2007). Finally, interviewers may reduce their use of trained techniques that are used less frequently during interviews (e.g., probing), especially experienced interviewers for whom training is more distant (Olson and Bilgen 2011; Tarnai and Moore 2008; van der Zouwen, Dijkstra, and Smit 1991).
Second, interviewers may learn to become more efficient at administering questions by reducing or eliminating seemingly extraneous behaviors, including stuttering or disfluencies while reading questions (Olson and Peytchev 2007). Interviewers may also reduce or eliminate extraneous laughter in an effort to shorten their interactions with respondents. They may do so because they place a greater premium on efficiency than rapport, or because their own enthusiasm conducting the survey wears thin over time (Cleary, Mechanic, and Weiss 1981; Houtkoop-Steenstra 1997). For the same reasons, interviewers may reduce their use of verbal pleasantries, personal disclosures, flattery, and digression. Interviewers may also reduce or eliminate task-related feedback (e.g., "let me just get this down") as early bugs in the interview hardware or software are corrected or as they become more efficient in navigating the interview system or entering responses. Task-related feedback may also be reduced if interviewers think it is not helpful for maintaining rapport or guiding respondents through the interview. Notably, these inefficiency-related behaviors may not be part of interviewer training, but happen as part of normal conversation.
Finally, interviewers may learn to increase the use of nonstandardized, time-saving behaviors such as changing the question wording (including making major changes or skipping questions), directively probing inadequate answers, changing answers when verifying them, and interrupting respondents. Although interviewers are specifically trained to avoid these behaviors, nonstandardized behaviors are ubiquitous in standardized interviews (Edwards, Sun, and Hubbard Chapter 6; Ongena and Dijkstra 2006). For instance, interviewers may be more likely to adopt practices, such as directively probing an uncodable answer, in order to advance through the interview more quickly (van der Zouwen, et al. 1991).
It is also possible that these behaviors will differ for landline versus cell phone interviews, as previous research has illuminated differences in interviewer and respondent conversational behaviors across these devices (Timbrook, Smyth, and Olson 2018).