MEASUREMENT STRATEGIES FOR EVALUATING COGNITIVE AND FUNCTIONAL PERFORMANCE
As discussed earlier in this chapter, indexing the impact of various behavioral interventions aimed at improving aspects of cognition or functional performance is a complex and multidimensional task and requires many decisions on the part of the researcher. These decisions are related to the type of measures to use, the number of measures, the timing of measurement, and staffing. Thus, measurement requires a careful conceptualization of the goals of the intervention and the possible benefits. There are multiple assessment strategies that can be employed and which vary in complexity, associated strengths and weaknesses, and challenges with respect to indexing the treatment goals and populations targeted by the treatment (Nuechterlein et al., 2004). In the section that follows, we briefly review the use of self-report and informant measures.
Informants are often queried about functioning, particularly in pathological conditions where the observer may have a long history of interacting and observing the cognitively relevant behaviors of the individual in question (Morris, McKeel, & Storandt, 1991). Similar to self-reports data, informant reports are subject to biases; it may be difficult to identify informants for all cases; and inclusion of informant data may add cost to a study. In some cases, it might be useful to gather data from several informants (e.g., clinician, caregiver, teacher) to gain insight into someone’s functional ability from different perspectives. Although this approach is useful, it can add to the cost and logistic constraints of the project and it may be difficult to integrate discordant ratings. Finally, another approach, which is based on observation, is to use informants who are unaware of the results of other assessment data and unmotivated to generate ratings that support either greater impairment (in search of disability) or reduced impairment (to reduce stigma). As described later, we have shown that only certain informants can provide information that meets suitable validity standards.
Overall, the strategies discussed earlier have different strengths and weaknesses regarding their reliability, validity, and practicality. A detailed discussion of these aspects of measurement is provided in the previous chapter. These strengths and weaknesses tend to be reciprocal in many instances; for example, more practical strategies may have some weaknesses with respect to validity. Thus, the selection of outcomes assessment strategies may be very different depending on the goals of the study and the populations to be assessed. The general criteria for selection of outcome measures are presented in Chapter 14. The focus of the remainder of our discussion is on performance-based assessment strategies.