Speed to Modern Tech Podcast Series, Episode 6

On this episode, we explore recovering from a ransomware attack and how businesses can strengthen their IT systems to protect themselves.

We divided the project into three recovery phases:

1. Business as acceptable. Enable the company to continue to conduct business, even if that meant temporarily using paper records.
2. Business as usual. Restore the company’s ability to conduct business with the same level of functionality it had before the attack, but with a more secure and resilient cloud-based infrastructure.
3. Business as transformed. Fully migrate to a cloud-first IT architecture to allow continuous compliance monitoring and provide the ability to reach from the cloud to remediate any future system issues.

Business as acceptable

To enable employees to conduct business as acceptable, our team focused first on the highest priority activities.

We provided paper templates to record transactions in a consistent and accurate manner so the information could be processed once replacement software systems were operable.

On the technology front, our first order of business was giving employees access to their data. Among other things, this meant ordering and configuring approximately 3,000 new laptops, which we were able to do by working closely with two computer manufacturers who were able to respond in a remarkably short period of time.

We also began replacing the inoperable on-premises IT infrastructure with a cloud-based version built on the Microsoft Azure cloud computing platform. Our first step was to recreate the company’s identity and access management systems to enable employees to log onto their software systems securely, which we accomplished by migrating them to the Azure Active Directory identify management platform and a Microsoft Office 365 tenant. To ensure security, we defined roles and permissions in a carefully crafted hierarchy. In the Azure administration portal, for example, which by default would give any administrator access to the entire infrastructure, we implemented multi-factor authentication and a “landing zone” designed to automate controls and enforce governance. We also required two or more simultaneous peer approvals to take any action that could compromise data or backups.

In concert with these efforts, a separate KPMG team focused on recovering as much data as possible from the client’s encrypted systems. Working closely with our key alliance partner, Microsoft, we recovered a surprisingly large amount of data saved primarily in file shares or development systems. Luck was a factor: we discovered one server that had been taken offline for maintenance just prior to the attack and were able to use it to restore the on-premises Active Directory service. By piecing together thousands of separate threads of information from hundreds of different sources we reconstructed most of the company’s key data. We then cleaned and organized this data to prepare it for import into the replacement systems once they were configured and available.

A pressing deadline

The company faced a pressing deadline during this first phase of recovery: the filing of its 10-K annual report with the Securities and Exchange Commission.

A 10-K details a company’s business and financial condition. To avoid reporting material risks or weaknesses — potentially exposing it to further attacks — the company had to quickly implement new security controls and prove to its auditor that it could successfully manage any further cyber assaults. With only six weeks until the audit, we recommended building controls into Microsoft Azure and storing the company’s critical data there. This approach worked, and the company was able to pass its auditor’s test and issue a clean 10-K.

Business as usual

For the next phase of the recovery, KPMG completed a secure restore of the company’s core software solutions, including its ERP, customer resource management, and human resources systems. They were built in the cloud and secured with Microsoft security features such as Single Sign-On, Multi-Factor Authentication, Web Firewalls, and Endpoint Detection and Response.

Business as transformed

The final phase of the project involved helping the client migrate its remaining systems to the cloud and managing both cloud and on-premises systems.

Using capabilities embedded in the company’s Microsoft 365 E3+ license, we implemented Azure monitoring, patch management, and MicrosoftSentinel, a cloud-native security information and event manager platform. Taking advantage of Microsoft Azure ARC, a set of technologies that bring Azure security and other cloud-native services to hybrid and multicloud environments, we also extended the capabilities of these tools to the few remaining on-premises systems. We sent all data to Microsoft Defender for Cloud to detect configuration drifts.

As part of this final phase we also helped update much of the client’s network infrastructure, including replacing outdated telecom provider circuits, re-architecting the network in Azure, moving the company’s virtual private network to Azure, and relocating processing-intensive and time-sensitive activities to data centers closer to where data was being generated or used.

In addition to guiding these software implementations we helped the client rethink its IT operations, beginning with development of an IT roadmap and project portfolio. We then helped the company create a new internal cyber security team complete with a staffing model and budget. We armed this team with the tools and processes needed to conduct both penetration testing and automated security audits.

During this phase of the project we also helped our client create a more robust disaster recovery framework that included provisions for handling any future ransomware attacks. Phishing was the most likely source of the original attack, and this new framework incorporated phishing testing capabilities to help spot any places where the company’s systems may still be susceptible to human vulnerabilities.

Finally, we helped the company create an IT architecture review board to guide future development efforts and establish the next set of priorities.

More than one infrastructure

As with many large organizations, our client had fueled its growth through acquisitions, swallowing as many as 60 smaller firms in recent years.

This had left it with a tangle of disparate IT systems that made the recovery effort significantly more complex. While addressing the most important of these subsidiary’s systems we designed and documented the processes we used to update them. The client’s internal team was then able to use these processes to take over the effort and safely integrate the remaining systems into the new cloud-based architecture. These same processes will act as the framework for integrating IT systems in future acquisitions.

We included people with experience and expertise in:

• Project management
• Disaster recovery
• Microsoft Azure
• VMWare
• Security operations

A key mission for the latter group: ensure the attackers would not be able to return.