Development and Implementation of Serious Games for Selection Purposes
Developing and implementing serious games for personnel selection requires adherence to the same psychometric and legal considerations as any other selection tool, but there are some unique aspects that also need to be considered. We have grouped these into the following categories: objectives, design and utilization. Each aspect is discussed in turn.
In order to design a game that is successful, game designers must first define what success looks like. The key objective that typically defines the success of any selection tool is how well it matches the needs of job performance criteria. More specifically, the stronger the correlation between candidates’ scores and job performance measures, the more successful the tool. There are other ways to validate selection tools (i.e., content and construct validity methods), but a criterion-related approach provides evidence of the utility of the tool with regards to its ability to predict job performance.
Complete coverage of methods for analysing the domain of job performance and determining which criteria are most critical is beyond the scope of this chapter, but we will highlight the main ideas in order to provide a basis for the sections that follow. In simple terms, the first step in the process is to identify what aspects of job performance should correlate with scores produced by the game. These aspects can either be subjective criteria
(e.g., supervisory ratings, customer satisfaction scores) or objective criteria (e.g., sales revenue, quality indices, production output). Once identified, these criteria need to be examined to determine which behaviours lead to successful job performance.
Once the behaviours have been determined, the game concept begins to take form. The behaviours form the basis for how the game is structured, specifically how these behaviours will be demonstrated during game play. The behaviours may be interpersonal (e.g., customer interaction, leadership, teamwork) or involve interacting with data or things (e.g., decision making, assembling parts, monitoring systems, safety procedures). It is important to understand not only which behaviours lead to successful job performance, but also which lead to poor performance.
Once the target performance criteria and related job behaviours have been identified, the next objective is to determine the measurement model. Will certain in-game behaviours lead to higher scores while others lead to lower scores? Will in-game behaviours lead to one overall score, or will sub-scores be produced? If the latter, how many sub-scores are needed? Will the sub-scores be rolled up into an overall score and if so, how will each sub-score be weighted? How will the scores be reported and what conclusions or interpretations should be made based on the results? Will a cut-score be required and if so, how will this be determined?
As with any other tool developed for use in personnel selection, a thorough job analysis is the foundation for answering these questions. Defining the job requirements in terms of type and level of KSAOs will inform the measurement model, which will in turn determine the scoring and reporting protocol. The validation and measurement objectives are the most critical elements of a successful game and serve to drive many aspects of design and utilization (see below). Much of the work involved in developing a game for selection purposes occurs before any coding begins and is time well spent once the game is operational. The technologies may be new and innovative, but adherence to sound psychometric principles and established procedures will lay the appropriate foundation to ensure the game provides a substantial return on the investment.