THE LIMITATIONS OF SELF-REPORT AS AN OUTCOME MEASURE
Objective outcome measures are important because self-reports in several domains of functioning are often unreliable. This is the case for both healthy and impaired populations. These domains include self-assessment of cognitive abilities, functional capacity, and some elements of everyday outcomes. Self-reports of previous experiences, such as health care and medical conditions, can be accurate in some circumstances, but even these reports become somewhat challenging if a long duration of time has passed since the experience. Healthy individuals often tend to overestimate their competence, and mood state variation can also impact the accuracy of self-assessment. Further, the discrepancy between self-reported functioning and objective outcome measures can provide valuable information about the response styles of an individual, with overestimation and underestimation of functioning having considerably different implications.
Often, people become candidates for interventions aimed at cognitive or functional enhancement interventions because of their subjective experience of cognitive change or difficulty performing everyday activities or because of concerns of a family member. Thus, self-reported measures of cognitive functions have been explored as an assessment strategy (Keefe, Poe, Walker, Kang, & Harvey, 2006; Ventura et al., 2013). Self-report measures of everyday functioning are often included in behavioral intervention studies aimed at improving cognitive or functional performance. An example of this type of measure is the Lawton and Brody Instrumental Activities of Daily Living (IADL) scale (Lawton & Brody, 1969), which requires participants to provide self-reported ratings on their ability to perform IADL tasks. The IADL domains included in this questionnaire consist of telephone use, shopping, meal preparation, housekeeping, laundry, transportation, medication responsibility, and finances. Another example is the Independent Living Skills Survey (ILSS) (Wallace, Liberman, Tauber, & Wallace, 2000), which assesses basic functional living skills and is typically used with patients with psychiatric disorders. Informants can also complete the ILSS.
Although these measures provide important insights into functional abilities and intervention outcomes, they obviously have both strengths and limitations. Overall, these measures are relatively quick and easy to administer and inexpensive. In addition, people have important insights into certain aspects of their behavior that are not accessible to outside observers. However, these types of measures are also subject to biases. As noted in Chapter 14, one type of bias is social desirability bias where people tend to report what they think the assessor or researcher wants to hear. In addition, people who are healthy with severe mental illness and those with other neuropsychiatric conditions all tend to overestimate their abilities (Bowie et al., 2007; Carone, Benedict, Munschauer, Fishman, & Weinstock-Guttman, 2005; Kruger & Dunning, 1999; Spikman & van der Naalt, 2010). Additionally, people with moderate or more severe depression tend to underestimate their functioning (Bruce & Arnett, 2004). Individuals who are experiencing subjective distress will often index their functioning in terms of their distress level and rate their functioning accordingly (Kaye et al., 2014). Furthermore, sometimes individuals have incomplete or inaccurate memories of their performance abilities.
At the same time, self-reported cognitive ability is typically unrelated to objective performance and the opinions of others (Durand et al., 2015; Keefe, Poe, et al.,
2006). It was recently shown (Keefe et al., 2015) that self-reports of cognitive functioning in people with schizophrenia were not sensitive to pharmacological cognitive enhancement, whereas both observer ratings and objective test performance were sensitive. Studies that have relied on patient self-report have also consistently found minimal correlation between reports of functional skills and objective indices of performance (Bowie et al., 2007; Durand et al., 2015; Sabbag et al., 2011). A reasonable conclusion is that the type of person who is targeted for cognitive or functional enhancement interventions is unlikely to be an adequate reporter of either his or her baseline functioning or his or her improvements from that baseline. Performance-based assessments are not prone to any of these limitations and have been shown, even in impaired populations, to be correlated with the achievement of functional milestones.