Cyber Crime: Print and Save This Article — You May Not Be Able to After Ransomware Strikes
By Clyde Hewitt
For The Record
Vol. 30 No. 6 P. 11
The purpose of this article is not to raise unnecessary alarm or overstate the risks of ransomware but rather to serve as a resource of the "dos and don'ts" for organizations that find themselves the target of an attack. Overall, the probability that a health care organization will experience a large-scale ransomware attack is relatively small. However, with the 424% increase in attacks between 2016 and 2017 (according to the 2018 IBM X-Force Threat Intelligence Index), the probability is increasing. In fact, according to the Beazley 2018 Breach Report, health care was the victim of 45% of all reported ransomware attacks in 2017.
CIOs who have endured an attack have been open about the early mistakes that came from the initial confusion and panic. A checklist outlining the dos and don'ts to follow during the initial stages of an attack can be helpful when navigating such a crisis.
Houston, We Have a Problem
Regardless of how an organization discovers a ransomware attack, everyone experiences at least some level of initial panic. The most important thing is to take a deep breath then initiate an incident response plan. Since ransomware strikes fast, organizations will not have time to start the planning process when the attack begins. Case in point: In one reported incident, the first infection compromised more than 2,000 servers within 14 minutes.
Once a ransomware attack has begun there is little value in discussing why expensive tools (eg, antivirus, firewalls, network access control) did not stop the attack. Save the analysis for later when the recovery is well under control. The only way to contain a ransomware attack is first to isolate the devices exhibiting signs of infection. This action can require shutting down virtual local area networks (LANs), buildings, or even entire facilities or campuses.
Some ransomware variants have been observed to first scan the LAN in an attempt to infect other devices before the ransomware starts the process of encrypting files. Therefore, there is a high probability that infected devices may exist outside of the isolation perimeters—even if the devices are not yet showing symptoms. It is better to establish a wide isolation perimeter than to have the attack spread outside of the initial perimeter and be forced to start over. Ransomware spreads exponentially, so response speed is the best tool.
When isolating devices, use lessons from a mass casualty event—specifically, the "triage tag." Under this strategy, any device suspected of being compromised is tagged and treated as such. Organizations will have time to come back and validate whether an individual device is compromised after the damage has been contained.
• Once an event is detected, anyone trained and authorized to establish a network perimeter should be empowered to do so without having to ask permission. Ransomware attacks rarely occur at 2 pm on a Tuesday, so be prepared to respond at a moment's notice.
• Initiate the incident response plan. Ensure that all participants are notified, including legal, the chief medical officer, facilities, physical security, public affairs, biomedical, pharmacy, laboratory, and all other covered entities that are part of the network. If the organization has ambulatory sites, those site leaders must be notified.
• Appoint a "recorder" to document all events, actions, and decisions in a detailed timeline. This individual should not be distracted with other duties.
• A ransomware attack is going to be an all available resources event, so initiate the phone tree. The impact of the event will be widespread and affect all departments and operations; therefore, the entire C-suite should be notified regardless of the time of day.
• If an incident response plan is not in place, establish one as soon as possible. A proactive plan is key to responding in a timely fashion and minimizing the negative impacts of an infection.
• Panic. Otherwise, the IT leader's value will quickly degrade, making him or her a liability to the organization.
• Let the chain of command delay the alerting and response process in the critical first minutes. It is OK for anyone to call the CIO or even the CEO directly to alert him or her of the issue.
• Think of the incident as only an IT problem. Ransomware has the potential to affect other areas such as biomedical, laboratory, radiology, pharmacy, and even facilities.
• Try to contain the attack alone. It takes many resources to contain the attack, making early alerting essential.
• Delay alerting the incident response team and the C-suite in order to implement downtime procedures.
The desired quarantine method is to identify the ports and protocols used by the malware to infect other devices and then those at all routers, gateways, and other network tools. Unfortunately, few organizations have an organic capability to perform the analysis during the attack. That being the case, these organizations should plan to leverage their extended incident response team, including the antivirus, network, and other security tool vendors. Organizations will receive a quicker response if they have previously executed contracts that include retainer language for incident response.
As soon as the organization has isolated the affected and suspect devices, the next step is to stop the outbreak by quarantining the entire environment. Start by protecting the most valuable data and systems at the server and storage level. The best time to consider a three-tier data center architecture has passed, so IT leaders should know ahead of time whether they have the capabilities and procedures for a bulk suspension (freeze) of all virtual servers and storage area networks.
If procedures are not in place for suspending these devices, a good alternative is a virtual or physical disconnect from the LAN until there is an opportunity to implement a more selective quarantine through port or virtual LAN management.
At the same time, disconnect any remote sites, partners, and other cloud services and let them know what is going on. Implement firewall rules or, if the organization does not have prepositioned scripts, consider pulling the external internet cables at least until the organization can get the new firewall rules in place. Consider white-listing as a short-term fix.
• Make it clear to the staff that they are empowered to initiate an emergency response in the event of a ransomware attack. Time spent seeking permission to initiate the obvious is counterproductive, especially when documented procedures exist.
• Develop clear instructions and checklists for how to isolate and quarantine systems. Print the lists and place them in known locations rather than relying on online systems that may not be available.
• Negotiate incident response contracts with key vendors and share the emergency contact information with key staff, who should store this information on a personal device. If the systems become inaccessible, access to online contracts, shared drives, or other key documentation may be delayed.
• Create a list of all external connections to the network and update it regularly. Include emergency contact information on how to reach external parties.
• Panic. Don't let others exhibiting signs of stress continue without intervention. Monitor performance and understand it is OK to send staff into a "time out" so they can realign their thoughts. A ransomware attack creates stress, which itself is infectious to others. Consider having counselors available.
• Think that any organization can recover from a widespread ransomware attack without outside assistance—it takes a team of internal and external resources. Alert the extended team early.
• Believe the IT budget will survive. For example, one midsize hospital used 60% of its total annual IT budget responding to a single ransomware attack.
By now the incident response team should be assembled and downtime procedures initiated. The incident response team should be capable of identifying all impacted processes, not just clinical services. Nonclinical processes such as timekeeping, human resources, patient financial, supply chain management, and facilities may also need to initiate downtime procedures.
Ransomware attacks will likely identify and infect all vulnerable devices within one hour. The challenge is to identify the compromised devices and start the remediation process. The sense of urgency at this point can be lowered to a manageable level.
The technical teams should focus on identifying the malware files, as well as the "index" or first machine infected. The malware files, along with the source IPs, should be shared with the various security vendors to allow them to update their antivirus and bot-net definition files to help prevent other infections outside of the original domain. The index machine should be preserved as is—hopefully by an expert in computer forensics. Remove it from the network and isolate it for future analysis by law enforcement and other third parties.
• Alert law enforcement—the FBI if the incident is large.
• Alert peer hospitals that may be willing to provide staff to help with remediation. Additional staff shortens the recovery period. Remember to formally authorize the external resources, a process that may need to be done manually.
• Understand that the IT leader's primary role is to support the teams emotionally and with substance. Consider catering healthy foods to support the around-the-clock work.
• Touch every device, including those in non-IT managed systems. There is evidence that biomedical and laboratory equipment can be infected.
• Bring up critical systems first, and connect by zones.
• Have a communication plan in place.
• Expect to recover quickly. A typical hospital technical recovery period extends beyond two weeks. It may be necessary to enforce mandatory time off to prevent staff burnout.
• Pay the ransom. Less than 50% of those that pay receive the unlock key. Ultimately, paying the ransom has no impact on the total downtime. It also encourages the perpetrators to attack other organizations.
• Forget that the organization is vulnerable to other attacks during the recovery period.
The last phase of the technical recovery is to monitor all systems to ensure they have been cleaned and patched. Anticipate that certain devices may have been offline during the attack, leaving them vulnerable to a reinfection. This is especially true of biomedical equipment, home health systems, and devices used by mobile members of the workforce. Keep tabs on devices placed with ambulatory patients who are still monitored through the network.
• Take time to document the event, paying close attention to the root cause analysis.
• Review the Office for Civil Rights' ransomware and HIPAA guidance, specifically as it pertains to the assumption that ransomware is a reportable breach.
• Conduct a full technical and process assessment of security, privacy, and compliance programs. Anticipate some type of inquiry by either the regulators or a third party looking at the event. The HIPAA Security Rule's standards for Security Incident Procedures, Response and Reporting (45 CFR under 164.308(a)(6)(ii) requires covered entities and business associates to "... document security incidents and their outcomes." In addition, 164.308(a)(1)(ii)(D) requires them to "regularly review ... security incident tracking reports."
• Assume that the event is over once IT has brought systems back online. It will take many months to enter the data manually from the downtime forms into the systems. This is a critical step to submit insurance claims and restart the patient financial process. There will be a significant cash flow shortage for many months unless external funding sources can be secured.
• Forget to reevaluate the limits of an organization's cybersecurity insurance.
• Think that everyone will sleep better at night. Large-scale ransomware attacks have the potential to create posttraumatic stress disorder symptoms.
Recovery from a major ransomware event will take at least several months. During the recovery period, health care organizations must bring in additional staff to transcribe paper downtime documents into the EHR. This will help recover much of the lost charge capture, but expect omissions.
CIOs should take time to reflect on their experiences, then pencil in suggestions and changes to this article. Post it on the wall for next time and hope it is never needed.
— Clyde Hewitt is vice president of security strategy at CynergisTek.