Improvement of Fire Statistics Continues Amid Controversy
features
“Leaders in the community just haven’t known enough to give a damn” about the fire problem, Joseph Moreland said last year when he was acting administrator of the United States Fire Administration (USFA). The reason was, he explained, that no reliable statistics were available to prove the size and seriousness of the total losses to fire. The situation is slowly being remedied, although not without differences of opinion over the results.
Fire service personnel long believed that the threat of fire was indeed worthy of more concern. But they couldn’t always convince the public, whose attention span on any subject tends to be short, or the builders of budgets, whose preference is for numbers over beliefs.
Although no fire fighter would attempt to attack a fire without tools, fire prevention and management personnel were in the position of trying to plan for and warn others of the scope of the fire problem without the tools they needed: supporting statistics.
Situation improved
Fortunately, the situation has improved in the last few years. Now thousands of fire departments are reporting extensive local data in a common format to the state level for analysis and forwarding to the National Fire Incident Reporting System (NFIRS) at the USFA. Also, numerous compiled statistics are published by the National Fire Protection Association based on a smaller sample.
Both organizations gather statistics in a number of fire-related areas, such as fire deaths, number and type of fires, and fire fighter deaths and injuries. For the purposes of this report, fire death statistics will be focused upon.
In any event, more fire information is available to fire departments for planning and education purposes. That is not to say those tasks are without any of the frustrations of the past.
Just as two mutual-aid fire engines would have difficulty relaying water at a fire if their hose threads failed to match, so too can difficulties arise when fire statistics from different sources don’t agree. For example, it would be unfortunate if a budget hearing for fire department funding became bogged down, deliberately or otherwise, by a debate over semantics or statistical techniques while the real issue of fire safety was forgotten. Yet it could happen if two sets of statistics don’t match and if the differences cannot be quickly explained.
Statistics don’t match
Fire death statistics, for example, from USFA and the NFPA don’t match. USFA reported a 1980 figure of 7600 deaths (down about 3 percent from 1979) while NFPA reported 6505 deaths (down 16.4 percent from 1979). That means USFA believes fire deaths were almost unchanged from the previous year, while NFPA believes that a significant reduction in fire deaths occurred. Overall, USFA indicated a 1980 fire death figure about 17 percent higher than NFPA’s. Is the difference important?
“As far as I am concerned,” says NFPA president Robert W. Grant,
“trying to reach absolute numerical agreement between NFPA and USFA begs the real issue. The real issue is that, although civilian deaths are down, too many Americans died by fire in 1980.”
The difference in statistics may not be a source of difficulty—unless it allows doubt to be cast on the reliability of both figures. To avoid that situation, anyone using either figure to illustrate the relative threat of death from fire would be advised to first understand how each figure was compiled. Only then can a reasonable justification be presented for using one of the figures, if a justification becomes necessary.
Even if only one organization reported a number of fire deaths, it would not be unusual for fire personnel to he asked how a statistic was compiled.
Consider any inquiry as an opportunity to add credibility to your presentation with a good, simple answer.
Estimates
The first thing to understand is that the fire death figures from USFA and NFPA are estimates, not absolute counts of each death. They are based at least in part on certain assumptions or “adjustment factors.” It is just not possible within existing budget limitations to document every fire or fire death on a national level. Estimates are used to good effect in any number of endeavors outside the fire service. But to contend that any national fire statistic is an absolute figure could cloud the issue.
Estimates can be very accurate or inaccurate depending again on how the estimate is made. On this matter both NFPA and USFA are very open about how their statistics are compiled.
An official death certificate is the foundation for USFA’s fire death statistics. When a person dies from fire, no law requires notification of a fire department, although one usually is called to the scene to extinguish the fire. But an outdoor or industrial flammable liquid accident, for example, could result in transport to a hospital and a subsequent fatality without fire department awareness. Or a responding fire department may not be motivated to fill out and send in a report. However, laws do require an official death certificate for every death.
“The Consumer Product Safety Commission here in Washington had a prior agreement with state health department to get copies of death certificates where some product was involved,” explained Bill Overbey, director of analysis and management studies at USFA. “We asked to plug into that existing certificate-review system, with fire-related causes separated for us.”
Limitations
If indeed every death certificate in the country could be checked, the resulting figure would be closer to fact than estimate, but some flaws exist which must be countered with adjustments.
For computer sorting purposes, causes of deaths on certificates have long been coded, but the system was set up before the particular needs of USFA were known. As a result, the codings for an auto crash involving death from an associated fire do not distinguish between a death just from the crash. In the same way, an arson-related death is not distinguished from another murder, according to Overbey.
Here is where an adjustment factor—8.3 percent of the other death certificate base—is added to cover fire deaths from these causes. That particular multiplier came from research on a smaller but verifiable sample of auto crashes and arson deaths, and the percentage was applied overall.
Another limitation is that not every state provided fire death information as expected. Death certificates must be filed, but the law does not require the USFA to be informed and grant money may not motivate. So, a best-guess estimate for those states was added, based on past experiences of a state along with actual figures from similar states.
The decision to estimate fire death figures for missing states illustrates the varying pressures on a data-gathering operation, which must work on a limited budget and seek cooperation from perhaps uninterested agencies while striving for accurate figures in a short amount of time. Trade-offs are made. The National Center for Health Statistics was described as a best source of statistics as far as accuracy is concerned. But such accuracy takes a long time, apparently, since that office has not yet reported various statistics for 1978. Not everyone wants to wait that long.
NFIRS network
In an ideal situation, the NFIRS network of the USFA would be receiving fire death information from every fire department in every state. Then fire loss statistics would again be less of an estimate. So far, however, only about 30 states are involved in NFIRS, and for various reasons not all departments in a state participate. Final NFIRS statistics must therefore be extrapolated for the entire country from a smaller sample, similar to the way NFPA statistics are compiled. For now, USFA feels the death certificate system is more complete.
Fire loss estimates from NFPA are “based on data reported to the NFPA by fire departments responding to NFPA’s annual fire experience survey and adjustments made based on data reported to the NFPA in the FIDO (Fire Incident Data Organization) System,” according to a report by senior statistician Michael Karter published in Fire Journal.
The fire experience survey was sent to a sample of about 7500 departments from NFPA’s list of 28,100, with a hoped-for response from 2800 departments. The target quantity was based on “the available manpower to process, edit and follow up on surveys.”
NFPA’s sample
Fifty-four percent of the selected departments protected less than 2500 persons each, but only 27 percent of them bothered to respond. Every department in a city of over 100,000 was included, but not all responded. Responding departments protected 34 percent of the United States population. The assumption is that the experiences of departments motivated to respond are the same as the experiences of departments not responding or not surveyed.
“I think it is significant,” Grand said, “that, although NFPA and USFA employ different statistical methods, our results are essentially the same.”
Gordon Vickery disagrees. He is the former administrator of USFA and now executive director of the Foundation for Fire Safety.
“Their (NFPA’s) assertion that there was a huge drop from 1979 to 1980 in civilian fire deaths isn’t so and they know it,” Vickery said in a press release. “They have blithely ignored data from USFA’s National Fire Data Center that disagree with their figures.”
One way to be more certain of statistical figures, according to Phil Schaenman, former head of the National Fire Data Center and now president of a consulting firm, TriData, is to make use of triangulation. In statistics, he said, that means taking several readings from several different perspectives in order to get the best fix on a subject.
He believes it is unfortunate for the fire service when figures showing a decrease in deaths are not confirmed through triangulation.
“Fire competes for national priorities,” Schaenman explained. “It’s hard to nail down support when the published figures seem to show a decrease in the problem.
NFPA feels that it calls them as it sees them, acknowledging that “around any estimate on any subject in any field, there are several types of possible error.
Verifying figures
Ten years ago, NFPA was grossly overestimating fire deaths in vehicle accidents, coming up with around 12,000 total fire deaths. Now, however, it makes a special effort to be accurate in an area also proving difficult for the USFA. For the current period, the NFPA attempted to verify all reported vehicle fire deaths by telephone followup.
Similarly, the USFA will try to verify its findings by comparing them with those of the National Center for Health Statistics (triangulation), starting with 1978 figures and moving to 1979 when they are finally released.
The NFIRS system can also be used, especially as the number of participating states grows. Its future and that of all of USFA, however, is under some cloud of uncertainty in expectation of more federal budget-cutting.
“It would be tragic if not disgraceful,” Schaenman said, “to abandon such a cost-effective effort. I would put the National Fire Data Center up for comparison against any other federal activity on a dollar-for-dollar basis.”
In any event, an ongoing effort is needed to improve the accuracy of fire loss statistics.
“How can the collection of fire death and injury statistics be made accurate? Grant was asked. One way is to have the fire service continue to improve their fire reporting through the uniform application of NFPA Standard 901, “Uniform Coding for Fire Protection.”
Meanwhile, fire service leaders should keep current with those figures that are available and should try to find more ways to use the information.
As Vickery has said, “Like bikinis, statistics do not reveal everything—but they reveal a lot.” □ □
