Executive Officer South County Fire Authority San Carlos, Calif.
Assessment Center Designed to Rate Battalion Chiefs on Handling Duties
An assessment center was specifically designed to rate five battalion chiefs after the decision was made to merge the San Carlos, Calif., Fire Department and the Belmont Fire Protection District and reduce the number of battalion chief positions to three.
The two fire services began soliciting proposals from managerial consulting firms in March 1979 to develop an assessment center to be held in May. The assessment center was specifically designed to rate the five battalion chiefs from both communities in a rank order system prior to July 1, when two fire departments were combined as the South County Fire Authority.
Chief Gary W. Schmitz of the San Carlos Fire Department was selected to head the newly created department, in this case the fire authority, in providing fire protection to an area of 10½ square miles with a population of 55,000.
Savings expected
The creation of the South County Fire Authority is expected to realize a $500,000 combined savings over projected budgets presented to the two city governments separately for the 1979— 1980 fiscal year. The savings were mainly due to the elimination of top management positions. The amalgamation made possible the elimination of one fire chief, one fire marshal, three battalion chiefs, two captains, and some rank and file positions.
Although most of the positions were eliminated through normal attrition, the number of battalion chiefs positions had to be reduced by other means, namely, the assessment lab. The reduction of management personnel and the reorganization of the two fire departments into one will improve the ability of the fire suppression forces to cope with emergencies in the two communities.
COGITAN, a Palo Alto consulting and research firm specializing in human resources management, was chosen to conduct the assessment laboratory for the selection of three battalion chiefs. COGITAN is led by Dr. Paul J. Hoffman, a measurement specialist who founded the Oregon Research Institute (ORI).
Although Hoffman provided collaborative support and umbrella supervision, Dr. Randall J. Strossen actually managed the assessment laboratory. Strossen received his Ph.D. from Stanford University in early 1979 and reflects the performance-based orientation of Stanford’s psychology department. This orientation (social learning theory) underlies the essential aspects of assessment laboratories and provides a ready complement to Strossen’s measurement expertise.
Sources of information
To determine the composition of the assessment laboratory, three sources of information were used. First, representatives from the Belmont and San Carlos Fire Departments were interviewed (total interview time exceeded eight hours). Second, official job descriptions and other published materials were studied. Third, an experimental task was devised to provide hard evidence for the final decision.
The third task involved translating the classification analysis of the battalion chief’s job into simple statements describing the required duties, abilities and bodies of knowledge. This produced a set of 25 items describing the battalion chief’s duties and a set of 30 items describing his required abilities and bodies of knowledge.
Each item was typed on a card and four Belmont and San Carlos fire service spokesmen categorized the two sets of items according to their relative importance. The resulting Q-sorts were subjected to statistical analysis. Without unnecessary technical elaboration, this technique establishes that the appropriate simulations were chosen (technically, this is called content validity).
Accent on management
The evidence indicated the primary importance of managerial tasks: planning, administration, policy formation, problem solving and organization. The battalion chief is also required to supervise, lead and maintain discipline.
To measure the abilities of the battalion chiefs, the following exercises were designed:
In-basket: This task simulated the candidate’s ability to handle the paperwork and make the decisions required of a battalion chief. This task included three policy formation and administrative tasks. Besides yielding information on the candidates as individual policymakers, this task yielded subscores on several key administrative skills.
Discipline vignette: Initial interviews, published material and the Q-sort data highlighted the importance of maintaining discipline. The discipline problem appeared to stem from the ability and willingness to take disciplinary action against old friends and co-workers—not from ignorance of proper procedure. Hence, the candidates had to model disciplinary action involving a friend—not only making and implementing a decision, but also justifying their actions, anticipating problems and proposing solutions.
Leadership measured
The need to establish leadership, work effectively in groups, form committees and develop joint projects was apparent from the interview, written and Q-sort data. To test these abilities, a leaderless group exercise was used. Individual and group goals were assigned and candidates were assessed on their individual and group competence. This task also included a measure of the candidate’s ability to use technical material.
A policy recommendation task provided the candidates with an opportunity to make recommendations directly affecting fire department operation. This task centered on a management problem whose solution required careful analysis and specific recommendations.
The interviews and official job analysis described the importance of handling public appearances and statements with tact. The Q-sort data, however, presented this ability in a less important light. Nonetheless, it seemed advisable to simulate this activity. To test each candidate for his ability to interact diplomatically with the public, individual on-camera interviews were conducted (public appearance simulation) and recorded on video tape.
Daily activities sampled
In addition to these simulations, three specific aspects of daily operation as a battalion chief were sampled and scored.
Although the entire process of developing, scoring and interpreting the results of the assessment laboratory depended on close liaison with fire department officers and other city personnel, the actual ratings were made by behavioral scientists.
The product of the assessment center was a set of scores and a written summary for each candidate and a rank ordering of the candidates for battalion chief in suppression, fire training officer and administrative officer.
These scores, and rank orderings, were intended to help officials make selection decisions and guide future fire department training programs. Strossen’s final report summarized how the assessment laboratory was developed and the technical (statistical) evidence for its design, scoring and interpretation.
