BY STEVE WINTERS
If you are involved in SELECTING safety equip-ment for your department, the task might seem extremely complex. The information presented here will help you make your way through a lengthy process, which may have seemed impossible at first, and will enable you to make well-informed and educated decisions. Boots are the example given in this article.
THE INFORMATION STAGE
Collect as much information about an item as possible from as many sources as you can. Include product manufacturers, component manufacturers, line firefighters, budget personnel, warehouse managers, and administrators. Include a wide and varied range of vendors, manufacturers, and distributors. (Note: You might be able to avoid the time-consuming process of conducting a field test by “piggybacking” off another fire department’s bid or purchase order. Piggybacking, however, can be a double-edged sword: You will have to live with the results of another department’s goals, testing, and analysis. This would be the time to determine whether this would be an option for your department.)
Determine your department’s specific needs by listing all criteria. Know exactly what you need the product to do before the field/wear test begins. At a minimum, consider ease of use, durability, price, comfort, availability, features, training, safety, and warranty.
Ask the manufacturers of the boots to be tested questions such as, What does “military grade” or “leather” mean? What is the difference between the same type of sole with different colored symbols? Can your department obtain any boot with a special type of sole or regular sole? How many sizes and widths of boots does the manufacturer carry? Are smaller, larger, and half sizes available? Are unusual sizes available, especially for smaller feet?
Use this information to set up your field/wear test. For instance, do you want leather or rubber boots? Do you need 14-inch structural firefighting boots, or will 10-inch boots meet your needs? Talk to line firefighters about issues such as tread design, traction, or combinations thereof. In addition, consider the features you needa breathable moisture barrier, for example. Determine which brand you want and why (the moisture barrier extends only halfway up the boot in some brands and runs the entire length in others). How does your choice interface with other equipment? As an example, you may need to consider zippers in the legs of your turnout pants if you choose 10-inch structural firefighting boots. Also, how will you ensure that no contaminants get inside your turnout pants when you slip the boots through? Another consideration is that it is very difficult to slip on 10-inch boots at night. These decisions will be the foundation for the field/wear test.
Use a minimum of three brands or manufacturers of similar items when doing a field/wear test. Additional brands/manufacturers should be tested, if possible. Know the differences between each brand, such as what type of thermal liner each has.
Before arranging for samples of the boots to be tested, have the suppliers measure the feet of each individual participating in the test. (It has been our experience that a one-half larger size should be used when members are changing from rubber to leather boots.) Using samples that have been physically sized instead of sized with a mechanical device decreases the number of problems associated with fit during the testing period. Use at least one individual who wears a size 6 and one who wears a size 13 or 14 shoe. It would also be helpful to use personnel who have problems with their feet or who use some type of orthotics.
Collecting Information
 |
Information on the tests themselves may be collected through a questionnaire (see Figure 1) on which likes, dislikes, wants, and needs are recorded. For the field/wear test to be successful, as many issues as possible must be addressed before the actual field testing. To ensure that answers are not prejudiced, ask open-ended questions, to elicit a response other than Yes or No. These questions should address the issues determined to be critical by your department.
Tracking System
 |
Develop a solid tracking system (see Figure 2). On your computer spreadsheet, include the name of the individual testing the item and his work assignment, shift, and phone number. Include also the item and the name and phone number of the product contact person.
Presentation/Documentation
Build a documented history of your results. Organize and store the collected information in some type of computer database. This record can then be used to present the selected item to management, educate purchasing personnel, train department members, and defend your choice to all parties. The documentation derived from keeping good records will be very helpful in convincing management and the purchasing department to buy the items your department has specified. Also, the more organized your process is, the better the chance that your department will acquire the needed test equipment at no cost.
Rating System
Developing a rating system will help expedite the decision-making process. The Denver Fire Department uses a rating scale of from 1 to 5, 1 designating the poorest performance level. Graph your results to see how each item performed in the test.
Assign a weight to each evaluation criterion. The results should reflect the most important needs. Use a multiplier to weight your results.
Example:
 |
In Table 1, ease of use, durability, and price were evaluated according to the priorities of a specific department.
- Ease of use was considered to be moderately important and, therefore, became the mean item. It was assigned a multiplier of 1.
- Durability was determined to be only half as important as ease of use and was assigned a multiplier of 0.5.
- Price was a significant concern of the department and twice as important as the mean. A multiplier of 2 was used.
The results are shown in the table. A graph can be created from the table to reflect the outcome.
From the table, we see that even though the Acme product was easier to use and more durable than the Universal item, it scored lower because of the importance of the price factor.
SETTING UP THE TEST
The greater the number of testers, the more accurate the test results will be. Select individuals who will give the item the most use. In this way, you will ensure that the equipment chosen will meet the needs of the majority of the department members. The field/wear test is the time for finding out whether a particular brand will hold up under your department’s use.
Give test participants adequate direction. Explain clearly what your goal is and how essential their input is to achieving a successful outcome. Be open and honest with all suppliers and people involved in the test, and don’t let your personal feelings affect the testing process.
Verify that all the materials used in the construction of the field/wear test item are made from similar types of materials so that you are comparing apples with apples. The ultimate challenge is to make your test as fair as possible. All suppliers will be trying to put their best product forward.
Distribute the spreadsheet/questionnaire to all fire stations and distributors/manufacturers. Update these sheets about once a month.
THE FIELD/WEAR TEST
Be thorough and consistent in your testing. If you select a person to test 14-inch boots, have him test all the 14-inch boots. Don’t mix in some 10-inch boots for him to test. If two people are to test brands of boots, don’t have one test X brand of boots and the other Y brand of boots. Have them both test all the brands incorporated in your field/wear test.
In our department, each member involved in the testing is given a cover letter listing the item to be tested and any special instructions. The letter also advises of the time line for submitting the evaluation sheets and for the running of the test. This step is important. Follow-up is necessary. After the first few weeks or so, test participants may become complacent and not turn in the evaluation sheets at the specified intervals. Meet periodically with all members involved in the wear test to review their evaluation sheets.
SUPPLIERS
When the item is placed in the field, have the manufacturers or suppliers of the item go out in the field and train and verify the fit and comfort of the test boot. Ask if there a local representative for each of the items being tested. If not, obtain the manufacturer’s toll-free number (if there is one).
Insist that the suppliers let you know if they change or alter a field/wear test item. Also, ask to be advised in writing which item(s) has been placed in the field. This information will help you to track field/wear test items and will provide proper documentation. Keep this information and the completed evaluation forms in your computer database. Make communications as simple and as thorough as possible, and keep all suppliers, department members, and supervisors apprised of the progress and test findings.
Give all distributors/manufacturers a copy of the filled-out evaluation sheets for their brands. Doing this helps them to keep updated on the test’s progress and provides helpful information for improving the quality of their gear. It also eliminates any surprises when the final decisions are made.
Give each manufacturer/supplier a shift calendar and a list of fire station phone numbers so they can contact the field/wear test personnel. Before you start your field/ wear test, put each supplier in touch with the purchasing departmentto ensure that it will be involved in the bid process. Give all suppliers access to your supervisor and the department chief. This provides them with several options if they feel things aren’t going the way they would like or if they can’t work with you.
THE SPECIFICATIONS
After all the data are in and all questions have been answered, evaluate the questionnaires. Have all the members who participated in the field/wear test meet to discuss the findings.
Determine whether the specification will be proprietary or more general so that several manufacturers can compete for the bid. If a specification is written for a particular brand, be certain that you can justify your results to the administration and the purchasing department. If you choose the proprietary bid, you may want to include several articles with your specification. One is the case of Whitten vs. Paddock (U.S. Ct. of Appeals, First Circuit, Sept. 6, 1974) in which the court ruled that proprietary specifications do not violate antitrust laws. The court also stated that professionals (specifiers) must make informed judgments on products they feel best meet their clients’ needs. Technically, few brands of equipment are exactly alike, and if the specifier wants to limit the specification to one source, he has the right to do so and to enforce it.
The courts also ruled that other manufacturers or suppliers can qualify as “or equal” only when the specifier permits them to bid. Where “or equal” is not stated in the specification, the specifying source still decides what products qualify as “or equal.” The court concluded: “The burden is on the supplier or manufacturer who has not been specified to convince the specifier that their product is equal for the purpose of a particular project.”
If you do your homework, you will find it is easy to defend a proprietary bid. Sometimes, it is helpful to include trade magazine articles or case histories from other departments (local, like-sized, and nationally recognized) about the items you are testing.
Check to see what prices other departments are paying for the same product and whether you can use the purchasing power of several departments to get a better price. Also, become familiar with the manufacturers’ warranty and replacement policies and the procedures for returning the product if that becomes necessary. You also may want to include the warranty in the specification itself.
Prepare the specification, and send it through the necessary channels for approval. Include all of your requirements on a cover page: time frame for delivery; care and maintenance requirements; contract violations and remediesspecific time lines so that all successful bidders understand requirements; decontamination and retirement procedure requirements; inclusion of all National Fire Protection Association (NFPA) requirements such as 1971, Standard for Protective Ensemble for Structural Fire Fighting, 1997, and NFPA 1500, Fire Department Occupational Safety and Health Program, 1997; and other applicable standards.
Send the final draft to the purchasing department for processing. For the selection process to effectively operate, you must have good communication with the purchasing department and the budget office. It may be helpful to show the personnel from those departments the entire process. We have had success doing this in Denver. (Be sure that your computer database contains field/test evaluations and copies of the specifications.)
The above test process has helped the Denver Fire Department get some of the best and safest equipment available in that the personnel who will be using the equipment made the selection decisions. This helps ensure end user buy-in, which is critical for success.
By gathering information and doing the necessary preparation, you will have answers to give the powers that be before they have to ask the questions. The more you can be involved with the process, the better your system will become. It never really gets any easier, but the rewards are greater.
Don’t get frustrated if things don’t go as planned. We in the fire service have learned that the best-laid plans can be changed at a moment’s notice. The field/wear testing process is no different. The work you do today will yield big dividends for your department. Our department has saved thousands of dollars and has acquired the best and safest equipment available.
STEVE WINTERS is a 16-plus-year veteran of the Denver (CO) Fire Department. For the past two years, he has been assigned to the Safety and Training Division and is responsible for research and development of personal protective equipment and most of the tools and equipment used on Denver apparatus. Previously, he was an officer on Tower One, part of the confined space, high-angle rescue, and collapse team unit.
