655 phone calls came into Bloomington and Normal emergency call centers during recent thunderstorms and high winds. The flood of calls overloaded dispatchers at certain points during the height of the disaster. The system was stressed but functioned as designed.
The McLean County Emergency Telephone System Board reports 655 calls went to Metcom, which is operated by the county, and to the call center operated by the City of Bloomington during the Aug. 13 storm. Some were to 911. Some were to non-emergency land lines.
Leaders such as Normal City Manager Pam Reece are praising the people staffing those centers.
“We are grateful to the dispatchers who were inundated with calls following Thursday's storm,” said Reece.
There are eight slots for dispatchers for the county at Metcom and seven in Bloomington. McLean County Board Chair Elizabeth Johnston said workers stayed past the end of their shifts to man all the stations.
"We also had a supervisor who jumped in on the phones and radios to handle the extra volume, and another supervisor who came in when he was not on duty," said Johnston.
The Emergency Telephone System Board oversees both Bloomington and Normal 911 operations. Brandon Lacey is the 911 system manager.
“Even with all those positions manned and the amount of phone calls that they got during that time, you cannot answer every single phone call,” said Lacey.
Lacey said that load is abnormal. Usually, the dispatch centers can handle calls easily with four to five dispatchers and not have what are called "abandoned calls."
"Abandoned call" does not mean the 911 system loses the calls. Lacey said there's a system called "automated abandoned callback."
"This system will go through and try and call each one of those numbers that didn't get answered, and it'll give them a message something similar to, 'You tried calling 911. Do you have an emergency? Press one,' and it puts them back into the 911 queue," said Lacey.
Not every call is equally urgent.
"Lots of times, the calls pertain to wires down, trees down, things like that. So, it's very much a team effort between the dispatchers taking the call, the public safety responders who are the first to respond and get there, and then the cleanup afterwards,” said Reece.
If the caller pushes 2 when the automated system calls them back and identifies their issue as non-emergency, the call is marked as handled. Sometimes though, the automated callback system can't get through to the person who called. The automation will try again. If it still can't get through or if the person on the other end hangs up on the automated system, that gets marked as a "priority abandoned call."
"And there's a distinct tone that is made on the phone system, and an asterisk on that queue saying there's something here you have to handle," said Lacey.
And then humans get involved again.
"They went through, called each one of those calls back to see if the person had an emergency, and to process and make sure that they got the assistance that they were needed," said Lacey.
Lacey said there were about 60 priority abandoned calls in Normal and the rest of the county, and 40 or so in Bloomington. All of those were handled by about 4 p.m. that afternoon.
Network outage
There was a minor wrinkle with Town of Normal information systems. The storm caused power blips early on, before major outages took hold. Town Innovation and Technology Director Vasu Gadhiraju said that prompted the town to shut down some of its computer servers.
"We have generators. We have UPS systems. We have everything. But the action that we have taken is a preemptive action. Nothing failed," said Gadhiraju.
She said when power availability changes abruptly between a utility company and generators and back again, there could be an interruption, and electronic equipment doesn't like that. The measure was supposed to protect the town network.
"If one of the systems are shut down abruptly we could run into data corruptions and some of the hardware issues that could take longer for recovery," said Gadhiraju.
The network shutdown cut access to the county's Computer Aided Dispatching [CAD] in some police and fire vehicles in Normal.
Johnston, the county board chair, said according to the county's IT department, the county Computer Aided Dispatch could not reach a Town of Normal server between 1:49 p.m. and 3 p.m. No other Central Illinois Regional Broadband Network secure tunnel circuit was down at the time.
The loss of that secure tunnel meant police and firefighters may not have had information that a dispatcher types into the electronic record but wouldn't provide over the radio while en route to a call for service.
Typing information instead of delivering it orally "is done to minimize the amount of airtime being used to dispatch a call when relevant call notes can simply be added to the call and read by the officer," said Normal Police Chief Steve Petrilli.
That kind of information is on top of the basics that do get radioed.
Connectivity depended on the type of connection a unit in the field had. Some phones and tablets could still have been able to connect, depending on the path the app used.
"The CAD mobile app is still in a limited testing phase at the town and has not yet been deployed for routine operations. ProPhoenix is a relatively new CAD platform. We are working closely with Metcom to fine-tune the system and make sure it meets our operational needs before expanding its use on mobile devices," said Gadhiraju.
It probably didn't make much if any difference.
"If I look at the types of calls for service that were being dispatched on Thursday afternoon as a result of the storms, most would have contained limited call notes or information. A tree blocking the roadway, a wire down, and/or flooded roadway does not have much to it, so the lower level or deeper information is limited at best. It would not have significantly impacted the officers' response on those types of calls for service," said Petrilli.
He said that situation is why there are backup systems in place so if a computer system goes down, they can use the radio. If radios go down, they have landline phones, cell phones, text, and so on.
As minor as it may seem, Gadhiraju said there's something to be learned from the town network outage.
"We are also going to look at cloud for additional layers of resiliency. So, if we need to spin systems up into the cloud quickly, it will allow us to do that," said Gadhiraju.
Standards and practices
Let's get back to the overall deluge of calls and the priority abandoned calls — the 911 callers that couldn't immediately get through to a human. The priority abandoned calls for Metcom were just under 17% of the total volume there. In Bloomington priority abandoned calls were more than 13% of the city total, said Lacey.
Do those numbers say anything about whether the system needs more capacity? After all, events like those severe thunderstorms don't happen every day. It may not be cost effective to build something that would be overstaffed.
NENA is the nonprofit professional association that sets standards for 911 operations. NENA determines staffing and call-taking positions based on a formula which creates a mathematical queuing model to calculate delays or waiting in queue based on traffic load, said Lacey.
The formula looks at total call volume and duration for all calls on 911, alarm lines, and non-emergency. With the recent addition in the past five years of Text-to-911, this is now also taken into consideration as it also ties up a telecommunicator, said Lacey.
One benchmark for centers to determine whether they meet objectives is that 90% of all 911 calls must be answered within 15 seconds, and 95% must be answered within 20 seconds during the busiest hour.
Another standard is called the Five 9s [99.999%]. Systems can't have downtimes more than 1 second per day, about 6 seconds per week, 26 seconds per month, and 5 minutes and 15 seconds per year.
Lacey said the system performed as designed. Johnston said those results are worth a shoutout.
"I would just like to take a moment to thank our Metcom 911 operators who have performed above and beyond their duties," said Johnston at last week’s County Board meeting.
Petrilli, from NPD, said the severe weather incident was "a good test" for emergency systems given the widespread power outage and the volume of calls.
"These types of incidents allow us to critically review our incident-response planning and more importantly the systems that control how we deploy resources in an emergency," said Petrilli.
With April tornadoes and a couple other high wind events this spring and summer, the McLean County systems are getting a lot of those tests this year.
