Imagine downloading malware from an attacker, but rather than delete it, you install it on your computer. Now imagine your IT staff installing that same malware on every computer, server, and system in your company.
Next, imagine opening your network to allow that malware to freely communicate back with the attackers for months, without interruption. Now, imagine that scenario 18,000 times, across a wide swath of industry and government agencies.
That was the SolarWinds attack.
Indeed, the SolarWinds cyberattack is one of the most sophisticated and broad cyberattacks in history and will likely be studied for years by cybersecurity researchers as a case study for a supply chain attack. Russian hackers infiltrated the computer networks of 18,000 SolarWinds customers, but it appears that their primary targets were U.S. government agencies including:
- U.S. Homeland Security Department
- U.S. State Department
- Nuclear research labs
- Government contractors
- IT companies
The overall planning, integration, and execution were carefully orchestrated by attackers of considerable skill and resource. The style and code structure indicate Russian attackers. The result was a very effective attack across thousands of large companies and government agencies.
SolarWinds is the best-selling management tool for large organizations and has over 200,000 customers worldwide. Of those, over 30,000 organizations downloaded the update immediately, and at least 18,000 instances were turned on to monitor IT systems. Attackers were then able to infiltrate any of these systems at will, completely undetected.
Cybersecurity leaders, managers, IT personnel, and especially students can learn a great deal about the sophisticated strategies of nation-state attackers. Smaller-scale attackers will likely use the same tools and tactics that were used in the SolarWinds cyberattack, so it’s important to be able to identify them and know how to combat them.
Organizations that don’t learn from the SolarWinds cyberattack increase the risk of becoming victims themselves in the future.
Why was SolarWinds chosen as an attack vehicle?
SolarWinds, based in Austin, Texas, is a leading provider of monitoring and analysis tools used in data centers around the world to monitor the health of systems.
SolarWinds’ Orion product serves as a collector and dashboard for IT professionals to check the status of networks, servers, and related systems.
For it to work, the Orion software is given access to almost all the systems in a modern IT infrastructure so it can collect and digest logs and alarms. That’s how the attackers got in.
Rather than attack organizations directly, the attackers used the Orion software from SolarWinds as a “Trojan Horse” to gain secure entry into the heart of IT systems. This is called a Supply Chain attack because SolarWinds was a trusted supplier to organizations around the world, including top-level U.S. government agencies.
What was the timeline?
Working backward from clues in log files and tools, experts (from FireEye, Crowdstrike, Kaspersky, and others) have examined forensic data to come up with the probable timeline for the SolarWinds attack. A high-level review of the timeline is a great way to begin studying and learning from it:
Sept. 4, 2019 — Attackers access SolarWinds’ network.
It is unclear how this was accomplished, but it was most likely through an email-based malware attack on a specific machine, which then compromised user credentials and allowed remote execution of malware. From there, the attackers carefully scanned the network and found the development environment where SolarWinds creates production code for its products.
Sept. 12, 2019 — This was not a random attack.
The attackers knew what they were looking for. At this point, the attackers had developed a small piece of code and inserted it into the Orion release code as a test. The attackers needed to know if their code could be embedded in the product and pass through the standard tests, checks, and digital signature process without being detected. The test passed when the attackers saw their code running in the latest Orion release completely undetected by SolarWinds and its customers.
Nov. 4, 2019 — Hackers continue testing their code.
For almost two months, hackers tested their code to embed in the Orion product. They accessed various DLLs (Dynamic Link Libraries) and files, the network, and read and write information. Once they understood the development infrastructure and the Orion code itself, they began developing their custom malware. Finally, hackers installed their code, called “Sunspot” by researchers, on the development servers that detected when a new build was being assembled. Its job would be to quietly inject the “Sunburst” malware into a product release.
Feb. 2, 2020 — Hackers inject malware into master Orion codebase.
The attackers successfully injected the Sunburst code into the master Orion codebase going out the door to Solarwinds customers. The Sunburst software was designed to sit dormant for up to two weeks to avoid detection and then called out to a Command and Control (C2) server for instructions. Later, researchers analyzed the code and noticed the malware is similar to the Kazuar strain of malware linked to the Turla group, a state-sponsored espionage group in Russia.
March 26, 2020 — SolarWinds releases compromised Orion updates to customers.
SolarWinds released its latest update (Hotfix 5 DLL) of the Orion software package to its customers around the world. This included both commercial and government agencies. Within about 2 weeks, the infected systems began reporting back to the attackers, asking for further instructions. At that point, the attackers quietly turned off access to targets they were not interested in, while starting wholesale extraction of data from targets of interest.
June 4, 2020 — Hackers accomplish their mission.
The attackers confirmed they had accomplished their mission and did not need SolarWinds any longer. On this date, the attackers quietly removed their code from the SolarWinds Orion product to cover their tracks. They also removed all traces they could find from network and server logs.
Dec. 12, 2020 — FireEye notifies SolarWinds of attack.
Over six months later, SolarWinds was notified of the attack by cybersecurity experts at FireEye, a Cybersecurity Solutions company. Because of their expertise, FireEye detected the attack on their own network and then notified SolarWinds so that remediation began across their customer base.
What did the malware code do?
Very simply, the Sunburst code was a stealthy backdoor access mechanism for the attackers. When activated, the code would send a DNS (Domain Name Service) query out to the Command and Control (C2) server set up by the attackers. Then it would wait for instructions. However, this DNS query was more than a request for an IP address.
It also used a domain generation algorithm (DGA) to encode the name of the victim’s network in the request. This allowed the attackers to know which victim network they had access to before responding. If a victim network was chosen for attack, the machine was instructed to download and execute additional penetration software.
Most often, this included commercial tools from Cobalt Strike, a company that specializes in tools for penetration testing. In every case, additional communications tools were added to provide a second means of access (i.e., a second back door) in case the Sunburst tool was detected and stopped. Cybersecurity experts call this “maintaining persistence” and is a valuable strategy for attackers.
The attackers took incredible steps to remain undetected. Here are a few examples:
Hackers were quiet and stealthy- Avoiding Communication Patterns
Once activated, the code waited a random period ranging from 10 days to two weeks before doing anything. This allowed the attackers to avoid correlations of new network communications with the Orion code installation.
Hackers Locked the Door Behind Them
As stated previously, the code used a DGA to create a unique DNS request when searching for the C2 server. In most cases, the victim was not an organization the attackers were interested in. The attackers responded with a special kill switch that not only deactivated the code but cleared logs, registries and deleted it as well.
By “locking the backdoor” behind them, the attackers could no longer get access to themselves. However, it helped prevent accidental discovery and collateral damage. (The Stuxnet attack of 2010 was detected when it escaped from the target network.) The attackers canceled attacks on almost all customers and generally focused on U.S. government agencies.
Hackers customized each attack to each victim
For those victims targeted by the attackers for the next phase, the Cobalt Strike packages were hand-crafted and customized to each unique network infrastructure. Not only did this help ensure success, but also made the detection of a common signature unlikely.
Each victim required customized remediation (Intrusion detection signatures, firewall rules, etc.). Once installed on the victim machine, the attackers took great care to not allow any associations between the new malware and the original code in the Orion software base.
Hackers “looked normal” - Exploited Older Domains and IP Addresses
The C2 servers themselves were older domains and IP addresses that had been established for some time. This helped hide them from DNS protection filters looking for access to new domains. Not only that, but the C2 servers had innocuous-sounding names and were on IP addresses in the United States.
Hackers Hid “Messages within Messages”
The communications between the victim machines and the C2 servers used the same protocols as the SolarWinds Orion software and appeared to be normal communications. In fact, the attacker messages were carefully encrypted and hidden in normal-looking messages (called steganography).
Hackers cleaned their fingerprints- Removed All Traces of Malware
After successfully gaining access to and downloading everything they wanted, the attackers went the extra step to clean up after themselves on the victim machines and networks. The hackers removed all traces of malware carefully, including backups, logs, and registries. As a final step, the attackers cut off their own access so it could not be traced back to them.
What can we learn? What should we do?
There are several lessons to be learned from studying the SolarWinds attack. First, it’s important to accept the fact that cyberattacks are inevitable. Eventually, an attack — especially a sophisticated state-sponsored cyberattack — will be successful. However, there are ways to make your infrastructure a more difficult target, and to detect and mitigate attacks when they do occur.
Therefore, what we as technologists and IT pros learn and do from here is critical.
Take these five steps to reduce and manage risk from cyberattacks:
1. Review your cybersecurity policies. Posture your organization against a well-recognized framework like the NIST RMF, Mitre Att&ck, and ISO 27000. These tools can help you figure out where to start by providing a blueprint of what a solid framework should look like.
2. Keep up to date on software patches. Yes, it is true that this attack was introduced in a software patch, but in general, software patches close many more vulnerabilities than they open. Once the attack was detected, SolarWinds customers who patched quickly reduced their exposure.
3. Segment your network. The Russian hackers moved through the SolarWinds network without too many hurdles; it was almost too easy. If you segment your network and limit administrator access to critical servers, it will make lateral movement more difficult and help prevent hackers from gaining entry to critical access points.
4. Use multi-factor authentication. Privileged accounts must be managed carefully. The SolarWinds attackers were able to compromise administrator accounts to set up the access they needed.
5. Learn to set up, maintain and use security tools effectively. Firewalls and desktop anti-virus provide only limited protection, so you will want to look into intrusion detection systems, identity management, DNS protection, data loss prevention, and many other security technologies. Subscribe to threat intelligence feeds such as Recorded Future or Talos to keep your tools up to date.
Continuous cybersecurity learning remains vital
To learn how to implement these precautions, it’s important to educate yourself and your staff on cybersecurity. All employees need exposure to basic cybersecurity hygiene and all IT staff must know the basic cybersecurity concepts.
Cybersecurity specialists need continuing education to keep up with the latest tools, techniques, and trends. Global Knowledge offers 70+ cybersecurity courses that help IT professionals stay current on industry best practices, concepts and developments. Browse the Cybersecurity
Course Catalog today. If you haven’t checked out Global Knowledge’s unlimited training subscription, you should. GK Polaris Discovery provides unlimited live-streaming virtual classroom and on-demand training which includes cybersecurity.
Every IT pro should at least be familiar with the domains covered in CompTIA’s Security+ certification. If you’re pursuing a career in cybersecurity, take the additional step of achieving the certification.