An Investigation of the Stability of Assessment Center Performance Consistency and Ratings of Job Performance

Abstract

The consistent lack of evidence for the construct validity of assessment center (AC) performance dimensions has led some researchers to propose alternative approaches to investigating candidate performance in ACs (Lance, 2008a; Lance et al., 2000). The lack of convergent and disciminant (i.e., construct) validity for AC performance dimensions has been referred to as “the construct validity problem” for ACs (Howard, 1997, p.21). The lack of construct validity evidence has been viewed as a problem by some researchers because the lack of convergent and discriminant validity for dimension ratings has traditionally been attributed to measurement error (Gibbons, 2007; Lance, 2008a; Lance et al., 2000). Recent research regarding the consistency of AC performance suggests that performance consistency may actually be a measurable individual difference where participants may reliably perform v differently in the individual exercises within ACs, also referred to as cross-situationally specific performance (Gibbons, 2007; Lance et al., 2000). Gibbons (2007) presented a performance consistency index to measure performance consistency in the context of an AC, which was adapted for the present study to measure dimension-level AC performance consistency. The results of the present study show that only overall consistency in an AC exhibits temporal stability in a sample of police officers who participated in a promotional AC at two separate points in time. Dimension-level AC performance consistency was not found to be stable over time. Additionally, it was found that AC performance consistency was not related to supervisor or peer ratings of job performance with the exception of a significant positive relationship found between overall AC consistency at time 2 and peer ratings of teamwork/cooperation. Lastly, supervisor ratings of job performance consistency were not found to be related to AC consistency at either time 1 or time 2.