Desktop version

Home arrow Mathematics

  • Increase font
  • Decrease font


<<   CONTENTS   >>

Results

The full model reduces the question-level variance relative to the base model by 87%, while the model with the individual question characteristics (that is, without the established tools) reduces the question-level variance by 86% (Online Appendix 18D, Table A18.D3). Thus, most of the question-level variation in RTs in these data is explained by this set of individual question characteristics. In the model that controls for the characteristics of respondents and interviewers, RTs are significantly different and longer for women, older respondents, and Latino respondents compared to other racial and ethnic groups (Online Appendix 18D, Table A18.D1); these effects remain largely unchanged in the models that also control for the question characteristics and established tools for evaluating questions (not shown).

IICs by Question Characteristics

TABLE 21.3

Median IICs by Question Characteristics (n = 102 Questions)

Question Characteristic

n

Median IIC

p-Valuea

Question on a complex topic

<0.001b

Yes

22

0.081

No

80

0.031

Question length

0.056'

Quartile 1 (<68 characters)

24

0.029

Quartile 2 (68-94 characters)

26

0.023

Quartile 3 (95-155 characters)

26

0.036

Quartile 4 (>156 characters)

26

0.052

Flesch reading ease score

0.134'

Very easy/easy/fairly easy (scores of 70.0-100.0)

55

0.031

Standard (scores of 60.0 to <70.0)

25

0.034

Fairly difficult/difficult/very confusing (scores of 0.0 to <60.0)

22

0.052

Question includes definitions/clarifying statements

0.243b

Yes

25

0.041

No

77

0.034

Question includes optional text

0.565b

Yes

52

0.034

No

50

0.040

Type of question

0.177b

Factual/demographic

85

0.031

Attitudinal/subjective

17

0.041

Question is deemed to be sensitive

0.584b

Yes

17

0.034

No

85

0.038

Source: National Health Interview Survey, 2017.

a An alpha level of .10 was used to determine if differences in median IICs were statistically significant.

b p-Value based on a Mann-Whitney-Wilcoxon test. c p-Value based on a Kruskal-Wallis test.

(Kruskal-Wallis test, p-value = .056). Collapsing quartiles one through three results in a median IIC of .032 (n = 76), which is significantly lower than the median IIC for quar- tile 4 (Mann-Whitney-Wilcoxon test, p-value = .011). We observe a similar finding for median IIC by the Flesch-Kincaid reading ease score. Collapsing the last four categories - "standard," "fairly difficult," "difficult," or "very confusing" items - results in a median IIC of .048, which is significantly higher than the median IIC (.031) for the "very easy," "easy," or "fairly easy" items at the .10 alpha level (Mann-Whitney-Wilcoxon test, p-value = .085).

Interviewer IICs by Interviewer Characteristics

As described earlier, 39 outcomes or questions were selected for the analysis of median IICs by interviewer characteristics. The overall IICs for these 39 items ranged from .0071 to .2113. Table 21.4 presents median IICs for each group of interviewers defined by each of four interviewer characteristics. For pace of interview, the fastest interviewers (group 1)

TABLE 21.4

Median IICs for 39 Items by Interviewer Characteristic

Interviewer Characteristic

Median IIC

p-Value*

Pace of interview (mean seconds per question)

0.0732

Group 1: <6.81

0.049

Group 2: >6.81 to <10.57

0.034

Group 3: >10.57

0.032

Cooperation rate

0.104b

Group 1: <64.02

0.047

Group 2: >64.02 to <87.85

0.039

Group 3: >87.85

0.020

Number of sample adult interviews

О

О

Group 1:1-20

0.047

Group 2: 21-40

0.038

Group 3: 41+

0.030

Worked on the NHIS in 2016?

0.208е

No

0.035

Yes

0.042

Source: National Health Interview Survey, 2017.

л An alpha level of. 10 was used to determine if differences in median IICs were statistically significant. b p-Value based on a Kruskal-Wallis test. c p-Value based on a Mann-Whitney-Wilcoxon test.

had the highest median IIC (.049) among the three groups, with groups 2 and 3 (slowest) having similar median IICs (.034 and .032 respectively). A Kruskal-Wallis test reveals a difference in medians significant at the .10 level (p = .073). For 22 of the 39 items, the fastest interviewers have the largest IIC. Collapsing groups 2 and 3, given their similar median IICs, and then comparing to group 1 results in a significant difference in medians at the .05 level (Mann-Whitney-Wilcoxon test, p-value = .023). Hence, interviewers performing at the fastest pace appear to be associated with a significantly higher median IIC compared with slower paced interviewers.

Regarding interviewer cooperation rates, we observe a consistent but nonsignificant decline in the median IIC across the three groups, whereby interviewers with the lowest cooperation rates have the highest median IIC (.047) and the interviewers with the highest cooperation rates have the lowest median IIC (.020). We collapsed groups 1 and 2 into a single category. This collapsed group of interviewers had cooperation rates of approximately 88% or less and a median IIC of .040 across the 39 items. When compared to the median IIC (.020) of the interviewers with the highest cooperation rates, the difference is significant at the .05 level (Mann-Whitney-Wilcoxon test, p-value = .035).

For the number of completed sample adult interviews measure, we observe a significant decline in the median IIC as interviewers conduct more sample adult interviews (Kruskal- Wallis test, p-value = .021). In pairwise comparisons, no significant difference in median IIC is observed between group 1 and group 2, but significant differences are observed between groups 1 and 3 (Mann-Whitney-Wilcoxon test, p-value = .022), and groups 2 and 3 (Mann-Whitney-Wilcoxon test, p-value = .013). To further underscore these findings, the smallest IIC was observed for group 3 interviewers (completed 41 or more sample adult interviews) for 28 of the 39 questions.

 
<<   CONTENTS   >>

Related topics