Sensitivity and Specificity
Sensitivity of a measure refers to the degree that a measure detects the presence of a characteristic in someone with the characteristics (e.g., cognitive impairment, high blood pressure, depression). Specificity refers to the likelihood that a measure will detect the absence of a characteristic in someone without the characteristic. The positive predictive value of a measure is the likelihood that someone with a positive result on a measure assessing a characteristic actually has the characteristic. The negative predictive value of a measure is the likelihood that someone with a negative result on a measure assessing a characteristic actually lacks the characteristic. Sensitivity and specificity may be combined to determine the positive likelihood ratio (LR) that describes the overall diagnostic properties of a measure (LR + = sensitivity/1 — specificity).
In addition to psychometric properties, there are other factors to consider such as the cost and feasibility associated with the measures of interest and the temporal relevance of a measure. It is important to ensure that the needed equipment and expertise for the administration of the measure and interpretation of the data gathered from the measure are available. Of course, available funding is also an important consideration. If one is interested in assessing both patients and their caregivers and participants are compensated for each assessment, the budget must include funds for both members of the dyads. Some types of measures such as physiological indices and biomarkers can be expensive and also require specialized types of expertise.
Other issues are related to floor and ceiling effects, both of which limit variability in responses. Ceiling effects occur when the items in a measure are “too easy” and floor effects occur when the opposite is true, the items in a scale or measure are “too difficult.” Other considerations are timing of the assessment, number of assessments, and participant burden. As noted earlier, mode of administration is also important. Assessors must also be thoroughly trained and evaluated periodically to ensure that they are adhering to the assessment protocol. “Assessor drift” is not uncommon if assessors need to administer an assessment protocol for large numbers of individuals over long time periods. Additionally, it is important to consider the amount of data that is being collected as well as protocols for data management, storage, and security. All too often these issues are ignored or considered after the fact. Finally, it is critical to consider the characteristics of the population in terms of characteristics such as ability, prior knowledge of a domain, literacy, and culture/ethnicity. We highly recommend pilot testing measurement instruments and data collection protocols with representative participants prior to engaging in formal data collection.