By John M. Buckman III
The logical structure of an individual’s certification evaluation requires an evaluator’s ability to distinguish between actual performance and a standard of performance.
When a student enrolls in a firefighter certification course, he must study the written materials and complete a set of practical skills identified in the applicable adopted standard. Competently evaluating an individual’s ability will ensure that the training process comes full circle.
If an evaluation process is to be delivered consistently, the use of checklists is a critical and required component of the certification process. This requirement sets the expectation that the evaluator will use the checklist provided to measure competence. It is also required that the evaluator fill out, sign, and give the checklist form to the student. The student then must return the skills checklist to the local fire department. The chief or designee must place the completed skills checklist in the individual’s personnel file. Why should everyone comply with this process? First, it is in most cases the law. Second, should a firefighter be injured or killed, this can demonstrate that the firefighter was properly trained and evaluated. This tool assists the evaluator, the instructor, and the chief.
WHY A CHECKLIST?
Checklists are intended as an evaluator’s generic guide for decisions typically needed when planning and conducting an evaluation. The checkpoints on the checklist are especially relevant when delivering an evaluation. Most of the practical firefighter certification skills require a demanding and complex evaluation.
A checklist is intended for evaluators who work under very different circumstances and constraints; the user must exercise good judgment and discretion in determining and applying its most applicable parts. It is a disservice to the student if an evaluator does not completely and fairly evaluate the student’s ability to complete the task according to the rules. When you don’t evaluate the student to the standard, you are telling the student he can do the job when in reality he may not be ready. This failure of the evaluator may come back to haunt that firefighter 20 years from now when the test is delivered on the street (at a real incident).
Conducting an evaluation requires just as much planning and preparation as delivering a course of classroom instruction. Remember, your role as the evaluator is not to teach or coach but to evaluate the student’s ability to perform the skill. Use the evaluation checklist in the delivery of the skills training portion of the class. This ensures that the evaluation process will complement the instructional delivery process.
OBSERVATION PROCESS: WHAT ARE YOU LOOKING AT?
Highly competent evaluators know observation skills are critically important to success. They work hard to develop their own observation skills, and they are accountable to the student and the state.
Leaders rely heavily on the observations of others to test their own impressions and require a body of knowledge about the issue at hand. Observation is learning on the flyit’s not something you sit down to doand every experience adds to your body of knowledge, leaving you a better evaluator. Competent observers need to process three items of informationperceptions, opinions, and factsobserved behavior that can be of help in your evaluation process.
If people are not effective observers/evaluators, often it is because they don’t observe/evaluate enough. In most of these cases, they don’t even know observation is a required skill.
If you want to be in the top 10 percent of whatever you do, work consciously and hard on the development of your observation skills. It will result in better firefighters who know the right way to do their job.
DEVELOPING YOUR OBSERVATION SKILLS
Learning how to “read” the student’s performance is extremely important to the evaluation process’ effectiveness. “Reading” a student’s performance means knowing the steps required to accomplish the task. Developing your observational skills helps the student make more sense of the theories learned from class. The best service learning experiences come when you work, critically and objectively observe, and then reflect on your experience. Using all of these stages throughout your fire-rescue service learning experience will increase your amount of learned information and improve your student’s ability.
You can better observe by doing the following:
- Increasing your patience to slow down and watch.
- Paying close attention to your physical surroundingswho, what, when, where, and how.
- Being aware of people’s reactions, emotions, and motivations.
- Asking questions that can be answered through observing.
- Reducing your prejudice to the skill standard. Many instructors don’t teach the skill exactly the way the standard states. This is based on their experience. In the certification evaluation process, your opinion does determine how the skill is performed.
- Being yourself.
- Observing with an optimistic curiosity.
- Being fair.
- Being ethical.
FOCUS THE EVALUATION
The first step in any evaluation is to develop a plan for the work to be done, called a “design,” and it is a particularly vital step to provide an appropriate assessment. A good design maximizes the quality of the evaluation, minimizes and justifies the time and cost necessary to perform the work, and increases the strength of the key findings and recommendations by minimizing threats to valid results. A good design consists of the following:
- Determine and clarify the object of the evaluation.
- Review the skill worksheet to understand the skill being evaluated.
- Review the course book to ensure that the book and evaluation worksheet match the expectation.
- Review any video or other visual medium that creates a model of how the skill is to be performed.
- Determine the number of students to be evaluated.
- Determine the number of evaluators needed to participate.
- Determine the location, tools, resources, and equipment needed to conduct the evaluation. This must be done for each skill to be evaluated.
Depending on the skills to be evaluated, the following are required planning:
- Location: interior/exterior, water supply, adequate space to conduct simultaneous evaluations, access.
- Vehicles: engines, trucks, tankers, rescues, specialized vehicles.
- Equipment: ladders, SCBAs, hose and nozzles, variety of tools.
The evaluator must plan for the process and must have things set up before the students who are being evaluated arrive. This means that the evaluator may have to arrive at the evaluation site two or more hours before the start of the evaluation process.
The ultimate goal of the evaluation program is to develop firefighters who understand and demonstrate competence now and in the future. Short-term outcomes of this program will result in changes in knowledge, attitudes, and skills.
Sound evaluations are grounded in clear and appropriate standards and criteria that are applied consistently and fairly.
JOHN M. BUCKMAN III has been chief of the Indiana Firefighter Training System since 2005. He has served as chief of the German Township (IN) Volunteer Fire Department and serves on the FDIC Educational Advisory Board and Fire Engineering Editorial Advisory Board. He served as president of the International Association of Fire Chiefs in 2001-2002 and is chair of the Program Planning Committee.
