Desktop version

Home arrow Mathematics

  • Increase font
  • Decrease font

<<   CONTENTS   >>

Difficulty Vignettes

Table 5.2 shows the mean interviewer ratings in terms of difficulty answering and asking questions about income and program participation across all six vignettes. Interviewers found the questions to be moderately difficult for respondents to answer, but slightly less


Mean Difficulty Ratings (Standard Errors in Parentheses) of the Income and Program Participation Vignettes by Respondent and Interviewer Perspectives (n = 25)




















Respondent perspective (answering a question)

1.1 (0.05)

1.9 (0.16)

1.6 (0.15)

1.9 (0.17)

2.2 (0.17)

1.4 (0.13)


Interviewer perspective (asking a question)

1.0 (0.04)

1.6 (0.16)


2.0 (0.19)

1.2 (0.10)

1.4 (0.13)


Notes: 1 to 3-point scale; higher scores indicate greater difficult)'.

difficult for interviewers to administer, consistent with the pattern observed in the previous vignettes on employment. The question about household rosters was rated as relatively neutral for both respondents and interviewers, although for some populations household rosters are perceived as sensitive (Tourangeau, et al. 1997). The remaining vignettes were rated as easy or neither easy nor difficult, but the question about expenses on healthcare products was rated as more difficult for respondents to answer than for interviewers to ask because the information was difficult to recall and calculate.

As observed earlier, interviewers reported being attuned to the fact that they cannot always predict how respondents will answer even seemingly neutral questions. Although interviewers are aware that unexpected situations can arise during the interview process, they do not always know how to respond in those situations. In terms of the questions about support payments, savings accounts, and spending on healthcare products, interviewers reported occasionally getting negative reactions from respondents on these questions for a variety of reasons. Like questions about income, these questions can also feel invasive to respondents: "I'd be worried that the fact of making support payments is sensitive. Sometimes people don't want to disclose the [payment] amount. For most people it's easy, get refusals sometimes;" and "Most of the time these questions are fine, but it could be intrusive, like we're checking up on them."

Although these questions were seen as sensitive for some interviewers, many mentioned during the interviews that the question about support payments in particular was difficult to administer due to the length of the question and having to read a large number of response options out loud:

"I'll get interrupted [while reading the response options]. I make the respondent wait to hear all of them because the answer is [the last response option listed] 'no payments made;"' "I would ask yes or no, line by line [for each response option). I'd anticipate some sensitivity on this topic;" and "The question is long, and the first option should be [the most common response) 'no payments made.'"

Interviewers recognized that the question was long and making respondents listen to a set of response options that may not apply to them could be tedious. Several interviewers noted that they would modify the question to filter out people who did not make support payments, or would change it to a yes/no format while reading the response options to help the interview flow better. For other questions, interviewers mentioned that problems might occur when respondents attempted to answer a cognitively burdensome question, such as the item about out-of-pocket expenses on healthcare products. This question could pose difficulty because respondents often do not know the information needed off the top of their head, or it may require complex calculations or estimation techniques such as addition, multiplication, or guessing the amount: "I have to probe this question when the respondent says, T don't spend anything on that.' Then I ask if they have allergies or take Tylenol and usually the respondent says yes. You have to help them add up what they spent. I help the respondent answer with memory probes;" and "Most people don't keep track of this throughout the year. They start thinking about all they buy, and trying to think of everything that falls under that, how often they buy it. If the respondent starts listing what they have, mentions the bottle size, I try to see how long it lasts them, figure out how often they have to buy them, for instance, once per month, every few months."

The question about biological children in the household seemed to be perceived as potentially sensitive and difficult for interviewers to ask. A minority of interviewers felt that this question was not sensitive because divorce is common, and that most respondents are open and forthcoming: "I try to make them comfortable, but difficult to change. If they aren't comfortable, I might say 'There are all different households these days;"' "This is uncomfortable to ask because there is a possibility the respondent will get offended. Don't need to label the child as adopted or biological. Makes people uncomfortable. I don't know why the survey asks about it;" and "I would show the screen to the respondent."

Finally, interviewers were asked about a question in which respondents are asked for their consent to use previously recorded information during a subsequent wave of the survey to shorten the duration of future interviews. Most interviewers recognized the value of asking for this, but felt that the question wording was long and awkward to administer: "Question is lengthy, the last sentence has two sets of parentheses;" "The question is long, kind of beats around the bush. A lot of people have commented that they'd like to do the survey online instead;" and "This question needs to be streamlined, otherwise people don't know what you're asking. I would read it as scripted but if I needed to probe, I'd say, 'This will make the [next] interview shorter.'"

<<   CONTENTS   >>

Related topics