How assessment centers work

My June article on career advancement, Strategies for answering situational questions, began a discussion on how to excel on promotional examinations that used an assessment center process. Since all my articles build on one another, I recommend you go back and review my June article briefly because it provides a comprehensive analysis of what an assessment center is, the “dimensions” assessors used to grade the various components of the examination process, and how the various “real-life” situational examination components are created.

One of the concerns in running an Assessment Center examination process is how to manage large numbers of candidates. It's expensive and time consuming when compared to traditional testing. Each candidate goes through from four to eight individual or group exercises, with a minimum of two assessors evaluating each (except for the exercise known as the "in-basket," in which there is usually only one). Since assessment centers often use "role players" as part of the exercises, more people are needed. The higher the rank the procedure is testing for (Lieutenant, Captain, Assistant Chief) the lower the number of candidates and the more manageable the testing process becomes.

It's been my experience that test validity and reliability begin to slip when large numbers of candidates are involved and scoring mechanisms are more likely to become a subjective rather than an objective process. Many cities use a hurdle approach by administering a traditional written examination and only allow candidates who achieve an artificial cut-off score to proceed to the actual assessment enter process. This defeats the purpose of using an assessment center in the first place and the examination process is further tainted by using an “in-basket” exercise (test) for those who scored high enough on the written examination to proceed to the assessment center phase.

Why? Because the "in-basket" is another form of written examination (although it does test for other dimensions) and now the parts and weights of the various components of the testing process are invalid. This is especially true if the department is unionized and promotional examination parts and weights have been made a part of the union contract.