New Research Report: What are security leaders saying about their security postures? View the Findings ➞

Top 3 Mistakes When Simulating Cyber Attacks

Breach and attack simulation (BAS) offers an efficient way to validate and test security controls, threat detection capabilities, logging levels in an environment, and incident response workflows. Simulating cyber attacks in this manner allows for security teams to proactively identify and remediate gaps; however, if not performed correctly, security teams may end up with a false sense of confidence. It’s important to avoid the three common mistakes below so you can maximize the value of cyber attack simulations for your security program.

breach and attack simulation to validate security controls

Mistake #1: Using logging levels not representative of environmental norms.

One of the biggest mistakes made when running breach and attack simulations is testing on a host that is configured differently for testing purposes. For example, if you increase the logging levels of Windows events or install additional endpoint products to monitor this host, this would not be representative of your environmental norm. Conducting testing in this manner can give the illusion that you have logging to detect the types of attacks across the entire environment, when instead that represents the best-case scenario. For the most valuable results, configure the logging on machines where testing is performed like other hosts of the same type.

Ideally, select an existing host in the environment or set up a new host as you would any other host in that area of the network. This can generally be accomplished by using the golden or master image for the environment, since that is what is being used to set up new devices.

Mistake #2: Testing a small subset of the environment’s attack surface.

Another common oversight in breach and attack simulations is to only run the agent on a single host and thinking that is the end all, be all for the status of an environment. Unfortunately, this can create blind spots as logging can differ based on:

  • Host Type (e.g. Servers vs Workstations)
  • Operating System (e.g. Windows vs Linux vs MacOS)
  • Network Segment (e.g. DMZ vs Internal)

Logging between servers and workstations typically differs, as logging levels and security controls are higher on critical servers. Testing only on a critical server could give the illusion that all hosts in the environment would detect simulations that were detected on this server. It is best to choose both a server and a workstation to perform testing on.

In addition to testing on both servers and workstations, it’s crucial to test on different operating systems, depending on how common they are used in an environment. Each operating system is different in the events that can be logged, scalability of logging, and how the events are logged, whether it be through GPO for Windows or through Auditd or osquery on a Linux host. Testing on any major operating systems in an environment is key to getting accurate cyber attack simulation results.

Finally, other log source functions such as firewall, proxy, and IDS logging is likely different for hosts in different subnets. In addition to this, results may differ if testing a remote or local host due to traffic not going through the same network security stack.  In light of these possible shortfalls, testing should be conducted on a wide variety of local and remote hosts that include different operating system types, servers and workstations, and different subnets of your environment.

Mistake #3: Simulating cyber attacks not representative of attacker norms.

Another mistake commonly made is simulating attacks that represent the exact conditions defined in SIEM or EDR detection rules. For example, say you are looking to test a SIEM rule looking for encoded commands run through PowerShell, where the criteria of the rule are defined by any command containing the string “-EncodedCommand”, where the process name is PowerShell.exe. If you only run the command including this string, we would miss many other attacks that employ the EncodedCommand parameter in alternative ways:

Using caret characters to escape character injection

powershell.exe –^e^C^

Truncated with alternate capitalized letters

powershell.exe –eNco

By using commands that an attacker would more frequently use, you may discover that your initial threat detection isn’t as robust as you previously thought and may want to address this by changing the rule logic to use a regex to detect the Encoded Command parameter. It’s best to test a variety of ways an attacker may perform an action during attack simulations, instead of being too specific to the rule logic set in place. Doing this will validate your current threat detection and advance new threat detection in your environment.

ReliaQuest GreyMatter offer cyber attack simulations to validate security controls

ReliaQuest GreyMatter provides integrated, automated cyber attack simulations to validate security controls and detection content.

By following these recommendations around attack simulations, you can increase confidence in your threat detection and logging in your network. Simplify this process even more through ReliaQuest GreyMatter’s integrated cyber attack simulations, which allow for the deployment of multiple agents across your environment to give you the assurance you need during testing and providing a substantial number of real-world scenarios.

Learn how to integrate attack simulations into your security operations strategy with the white paper: Continuous Attack Simulations: How to Identify Risk, Close Gaps, and Validate Your Security Controls

More Articles

5 Ways to Use Continuous Attack Simulations to Validate Your Security Controls

How confident are security teams that their controls will catch attacks when they arise? Ask around, and you might notice a theme: as enterprise security models grow in complexity, teams struggle to validate their security controls, increasing the likelihood of undetected breaches, gaps in protection, and weaknesses from unpatched systems. These scenarios are indeed worrisome, […]

Best Practices for Detecting 5 Common Attacks Against Kubernetes

Container orchestration tools have risen in popularity within the past few years, especially open source tools like Kubernetes, and have enabled organizations to more efficiently deploy and manage applications. However, these tools also come with their own security risks. All tools are susceptible to misconfigurations and insider abuse, in addition to more serious vulnerabilities that […]

Detect Solorigate and SUNBURST Attacker Techniques with the MITRE ATT&CK Framework—We’ll Get You Started

As folks are continuing to work to address the Solorigate/ SUNBURST compromise, our team has been mapping the tactics and techniques used by the attackers to the MITRE ATT&CK framework, and building detection content to deploy for our customers.  If you haven’t already, please read this blog first to get the basics. What follows is […]