Virtualization for Defense Evasion: Hiding Tracks

Cyber-threat actors are agile, resourceful, and consistently adapting their techniques to remain undetected. Those using virtual machines (VMs) for that purpose can enjoy enormous success. Even if activity is detected—such as lateral movement via remote desktop protocol (RDP) or ransomware staging through Server Message Block (SMB)—a proper root cause analysis can still remain elusive, as can tracing the activity back to an original compromised host.

During an active network intrusion, a responder’s objective is mapping all internal assets accessed by the attacker, to identify the original compromised host and full scope of activity, including method of initial access. The responder then evicts the threat actor by isolating infected hosts and cutting off command-and-control (C2) channels, among other actions. Unless activity suggests an internal host that just doesn’t seem to exist.

This opens up an opportunity for post-compromise threat activity for the hidden threat. And a particularly inviting avenue is provisioning a virtual guest operating system on the compromised endpoint. Let’s follow the hidden tracks, to see how the route can be blocked.

Virtualization Attack Chain

Figure 1 shows the basic steps (not in any set order) needed to take advantage of virtualization for defense evasion.

Figure 1: Threat actor kill chain

Starting out the chain, the threat actor needs access to an operating system image and a hypervisor from the compromised host. The image file would be used to install an operating system on a newly created virtual disk, and the hypervisor would be used to create that virtual disk and perform the installation. We recently observed potential file downloads in one case that led us to base the telemetry source for this phase on wide area network (WAN) traversal monitoring technology—most often, forward proxy logs.

Image Ingress: Devil in the Download Details

Although detecting downloaded disc image files isn’t a foolproof endeavor, one method is checking for certain extensions in URLs, such as .iso and .img. This will detect image file downloads from benign sites, such as Ubuntu, and can also spot image file downloads from unexpected locations, such as Amazon S3 buckets. However, attackers are more likely to preconfigure an operating system image with tools ahead of time, so likely would not download benign operating system image files. It is also unlikely that attackers would leave standard image file extensions for defenders to identify.

Another method for detecting image file ingress is using the MIME type provided by the proxy logging. Although there are related MIME types, ReliaQuest found that image file downloads were typically assigned the application/octet-stream media type. This type is a catchall to files that do not match any other type.

But even if image files consistently matched MIME types specific enough to isolate them, you still might hit a dead end: The threat actor need only compress, encrypt, or otherwise embed or hide the downloaded file to avoid making their image file downloads so obvious.

What’s a Hypervisor?

Any software managing the subdivision of physical resources among virtual machines (VMs) is called a hypervisor. Type 1 is known as bare metal, and is installed directly on the hardware. Type 2 is considered hosted, and is installed on top of an existing operating system.

Hypervisor Hyper-vigilance

With an image file successfully downloaded onto the compromised host, the attacker installs a hypervisor. Few corporate workstations are likely to have the third-party hypervisor application. Detecting the download of hypervisor installer files is a simple matter of looking at specific binary names or specific websites, such as hxxps://www.virtualbox[.]org/wiki/Downloads. Host-based logging and application inventory software can also be used.

Building a query based on command-line logging or simple execution of an installer file isn’t optimal for confirming the installation of unauthorized hypervisor software; the query doesn’t confirm the installation actually took place, and the threat actor could simply rename the binary to avoid detection. A more optimal method would be to use telemetry associated with the installation itself such as service installation logs, registry keys creations, or MSI Installer logs.

Virtual Machine: Configuration and Control

When creating the new VM, the attacker needs some way to specify its configuration, such as how much RAM or disk space it should have or whether it should share a clipboard with the host machine. Hypervisor applications help with this; some require graphical configuration, and others allow the use of the command line to specify various settings. The latter option can reveal relevant configuration details, such as whether they enable RDP for the VM.

Now the threat actor needs a way to control the operational VM. They could simply use RDP to access the host machine or even navigate directly onto the guest VM and control it graphically. Or they could use the command line to do it. Some hypervisor software offers the ability to send commands, originating from the host machine to the guest VM for execution.

Detection rules can be based on each hypervisor’s method of allowing command-line management of a guest VM. In this way, signs of post-compromise tool usage can become visible. There are additional ways threat actors can configure guest VMs, such as preconfigured operating system images or VM application files, which would allow them to set up their control ahead of time, such as by setting up a pre-installed beacon implant.

VMs Running Rogue: C2 Traffic

At this stage, the threat actor may have an operational guest VM, which they can control directly via C2 implants. That would bypass any host-based searches that may look for signs of VM control via command-line logs. The only remaining telemetry at this point would be the C2 traffic itself.

We’ve run through some potential methods to detect C2 traffic and prove it was generated by a VM. This included installing test beacons on VMs created by various popular hypervisor applications. Although results varied based on the type of hypervisor, we directly attributed actions of a guest operating system using host-based technologies, such as operating system logs and EDR.

More specifically, when Type 2 hypervisors are used, their activity is hosted by the host machine’s operating system. This exposes any action taken by the guest VM, which requires physical resources to be provisioned by the host operating system. For example, if the guest VM needs to make a network connection, it needs the use of the physical NIC, which is controlled by the host operating system. This makes the host operating system aware of any network connections originated by the guest VM.

In practice, what this looks like is either Sysmon network event logs or EDR logs attributing network traffic to hypervisor application processes. This typically will look different for each hypervisor implementation.

Figure 2 shows an example of a ReliaQuest test to see what level of visibility could be gained of the network operations of a running VM. The red boxes indicate our test C2 traffic, as viewed from the network point of view through Wireshark. In this configuration, we had NAT mode enabled, so the source IP address belongs to the host machine.

Figure 2: Network connection from staged VM to C2

Figure 3 shows a Windows VirtualBox VM with the beacon installed; on the left is our C2 server hosted in AWS. The screenshot shows that the beacon was used to start an interactive shell on the VM, demonstrating that an active network connection existed between the guest operating system and the Amazon Web Services (AWS)-hosted C2 server.

Figure 3: Demonstration that an active network connection existed between the guest operating system and the AWS hosted C2 server

Figure 4 shows a command prompt window present on the host machine, indicating the active network connection established with the C2 server.

Figure 4: Active network connection to C2 server from a process with a PID of 12640

Figure 5 shows that the process ID associated with the network connection correlates with a VirtualBox VM process.

Figure 5: VirtualBoxVM.exe assigned PID 12640

Forensic Lessons Learned

Linking the available host telemetry to other log sources, such as network and Windows logs, enables analysts to identify certain actions—such as RDP connections, network scanning, and Windows logins—being sourced by VMs. Potentially, it can even link it to a specific VM, although only when Type 2 hypervisors are used. Rogue VMs created by Type 1 hypervisors won’t be visible, because they have direct control over the physical hardware. A primary example is Microsoft’s Hyper-V.

To mitigate, hunt for threats in your own environments, and establish a baseline of expected virtualization use to look for anomalies. This could include hypervisor use on unexpected devices, or even unusual configuration changes. Organizations can also block unauthorized hypervisor software installations on endpoint devices. Whatever the mitigation, the real lesson here is that defense evasion via virtualization is a valid technique that carries real risks.