The Role of Simulation Exercises in Selection
Ryan S. O’Leary, Jacob W. Forsman and Joshua A. Isaacson
Simulation exercises are a group of assessment methods that measure applicants’ workrelevant performance while performing tasks, interacting with others or using equipment or technology (Callinan & Robertson, 2000; Ployhart, Schneider, & Schmitt, 2006). This includes a range of assessment procedures, all of which closely resemble actual parts of the focal job for which the applicant is being assessed, including situational judgement tests, assessment centre exercises and work-sample and performance tests. Across assessment types, simulations can be designed to measure a wide range of constructs such as hard and soft skills, personality, task performance, job knowledge and cognitive ability (Tuzinski, 2013). They have been used in selection systems for a full range of jobs and positions.
Unlike traditional assessment approaches which rely on indirect evidence of applicants’ skill and ability to perform, simulations rely on direct evidence. The underlying premise of simulations is the idea of point-to-point correspondence (Asher & Sciarrino, 1974); that is, prediction is improved to the extent that the assessment mirrors the criterion domain for a given job, focusing on some or all of the behaviours required for successful overall performance. This approach is different from that used by measures such as assessments of personality and cognitive ability, which focus on applicants’ predispositions to behave. Simulations rely on samples of behaviours to predict subsequent job performance. Selection decisions are most accurate when based on behavioural consistency, the notion that the best predictor of future behaviour is past behaviour in the same or similar contexts. Simulations maximize the prediction of job performance by evaluating behavioural samples that match job performance requirements rather than signs (Wernimont & Campbell, 1968).
Simulations have a long history of use across the human capital lifecycle including personnel selection, certification and training programmes, as well as in a number of educational contexts. In personnel selection, the use of simulations dates back to their use
The Wiley Blackwell Handbook of the Psychology of Recruitment, Selection and Employee Retention,
First Edition. Edited by Harold W. Goldstein, Elaine D. Pulakos, Jonathan Passmore and Carla Semedo. © 2017 John Wiley & Sons Ltd. Published 2017 by John Wiley & Sons Ltd.
in the military in the 1940s and the start of widespread organizational use with managerial assessment centres at AT&T in the 1950s (Bray, Campbell & Grant, 1979). Advances in technology are leading to increases in simulation sophistication, realism and fidelity and have made development and administration more economical. The once static and paper- based simulations of years past now include multimedia and leverage technology for administration and scoring. This has provided endless possibilities for development and measurement.
Today, simulations feature prominently in many selection systems across industry sectors in private and public organizations around the world. According to an international study conducted by the Corporate Executive Board (CEB) (Fallaw & Kantrowitz, 2013) on assessment trends, 67% of the companies surveyed use some form of simulations as part of their hiring process. Of these, 42% were from China, 13% were from South Africa, 11% were from the United Kingdom, 8% were from Australia, 8% were from the United States and Canada, and 18% were from other countries. This trend is likely to increase because of the number of benefits simulations provide over traditional approaches for predicting job performance, including practicality, criterion-related and incremental validity, potential for smaller subgroup mean differences and less adverse impact, and positive applicant reactions and engagement. Simulations remain the only assessment type that can simultaneously measure the interaction of constructs required for job performance.
The value of simulations is well understood and based in sound science. However, as technology advances our ability to develop simulations at an accelerated pace, a gap between research and practice has started to open. In this chapter we review the current research and trends related to the use of simulations in personnel selection. We begin by providing a taxonomy of simulation types, followed by a discussion of simulation fidelity and its impact. Next, we review the psychometric evidence supporting their use to include validity and subgroup mean differences and discuss issues related to construct measurement. This is followed by a discussion of applicant reactions, cross-cultural considerations and the important role simulations can play in recruitment and organizational branding. We conclude with a discussion of topics in need of consideration by simulation developers and users and we project the future of simulations for selection.