Impact of fidelity on validity
While a more detailed discussion of simulation validity will be presented later in this chapter, the impact of fidelity on validity is worth mentioning here. Under the assumption of point-to-point correspondence, one would assume that higher levels of fidelity would translate into higher observed validities. However, it is not always the case that correspondence between the predictor and criterion results in an assessment that is more job-related, and in practice the level of correspondence needed is not always straightforward (Callinan & Robertson, 2000). Interestingly, simulations across the fidelity continuum have exhibited similar levels of validity in individual studies, suggesting that moving to low-fidelity simulations may not always impact validity (Lievens & Patterson, 2011). This has led some to argue that, based on the current and limited understanding of fidelity and validity, the time and resources required to develop high-fidelity simulations may not be justified.
In their review of the validity of simulations ranging from low to high fidelity, Boyce, Corbet and Adler (2013) correctly assert that while various simulation types generally have exhibited comparable levels of validity across individual studies, same-sample studies directly comparing simulations ranging in fidelity are lacking. It is frequently argued that the impact of fidelity on validity needs to be examined in the context of both stimulus and response fidelity. For example, Lievens, De Corte and Westerveld (2012) examined how response fidelity impacted job performance. They found that for police trainees, only an actual behavioural response led to significant predictions of future job performance. Funke and Schuler (1998) found that increasing SJT stimulus fidelity had no impact on validity, while increases in response fidelity (multiple-choice, written, oral) were associated with greater validity.
Despite these studies, research examining the impact of fidelity on simulation validity is needed before definitive conclusions can be made. This is particularly true as advances in technology are continuing to accelerate our ability to vary stimulus and response fidelity. To date, no clear guidance has been provided on the level of fidelity required to achieve desired levels of validity, and this is a clear area of need when considering costs and utility. As research accumulates, frameworks can be developed that will guide simulation development. What makes researching fidelity difficult, beyond the distinction between stimulus and response fidelity, is the fact that fidelity’s importance is likely to vary according to job.