The Security Metrics CISOs Use Every Day

In the dynamic world of cybersecurity, where threats are ever evolving, the role of a Chief Information Security Officer (CISO) is essential in protecting the business. Not only must CISOs establish programs to do this, but they must also effectively measure and communicate the value they bring to the organization. Using the objective “metrics that matter” supports this mandate. In this blog post, we will explore the practical security metrics CISOs use every day and how they use them to demonstrate their impact on the business. 

Seeking the Story Behind the Numbers

As with any leadership role, CISOs must also justify their program spend by showing program progress and corresponding risk reduction over time.  When they use the proper security metrics, they can more effectively tell that story.  

CISOs have an abundance of security tools at their disposal, each providing a different capability within their security program.  However, relying solely on individual tool outputs can be limiting. It’s like looking at a pixelated picture, where each tool represents just one pixel. To truly understand the bigger picture, CISOs need to stitch these pixels together to form a coherent narrative—a story. 

In this blog, we’ll share the big picture of the security metrics CISOs use every day, starting at the top and zooming in. First, we’ll discuss the metrics that CISOs use at the strategic level, then the operational level, and finally at the tactical level. 

Strategic Security Metrics

A CISO’s role extends far beyond the walls of a security operations center. Strategic conversations with the C-suite executive team, audit committees, risk committees, and board of directors are integral to aligning risk with business objectives. 

Engaging in strategic conversations with diverse stakeholders requires adaptability and nuance, as each enterprise has its own unique set of priorities and driving factors that provide context to raw numbers. The most effective CISOs tailor their communication to a specific audience. For example, discussions with the C-suite may need to focus on the potential impact of security incidents on the organization’s reputation and financial stability. In contrast, meetings with risk committees may require a more analytical approach, emphasizing the identification and mitigation of emerging risks.

Depending on who’s in the room, a CISO might cite: 

• Risk acceptance vs. risk deference
• Revenue, budget, and top- and bottom-line numbers
• Something like a security "credit score," to showcase the program’s overall state and progress over time

An essential component of these conversations involves analyzing security incident trends. CISOs often present overarching trends to showcase the organization’s security posture. For example, an increase or decrease of true-positive incidents on a quarterly basis provides valuable insights into the effectiveness of security measures and helps drive decision-making at the organizational level.

Operational-Level Security Metrics 

Outside the boardroom, a CISO manages the operational aspects of an organization’s security program, requiring them to consider different metrics. At the operational level, decisions revolve around strategic investments, program maturation, and aligning security initiatives with the organization’s roadmap

Some of the metrics that CISOs lean on for clarity at the operational level include:

• Visibility and detection coverage: It’s important to consider these two metrics in tandem: Visibility does not equal detection. You may have an EDR agent employed across your environment, but if you never deploy detections within that EDR agent, you'll miss critical alerts.
• Compensating controls and prevention capabilities: Operational-level metrics also assess the effectiveness of compensating controls, which act as backup measures when primary controls fail or are bypassed. Assessing the ability of compensating controls to effectively mitigate risks is critical. Additionally, measuring prevention capabilities helps determine the effectiveness of security measures in stopping potential threats before they cause harm.
• Program maturity: Metrics like mean time to resolve (MTTR), threat hunting activity, and use of automation provide insights into security program maturity. MTTR at the operational level helps CISOs assess the health of their security operations: An increasing MTTR may indicate potential resource constraints or training needs for the security analyst team. Understanding successful or partially successful threat hunts helps evaluate program proactivity while automation through functionality like playbooks can help measure program efficiency and effectiveness.

On a CISO’s top-10 priorities list, there are always 11 items to cross off. These operational-level metrics can help CISOs identify the areas that offer the greatest return on investment in terms of risk reduction and overall program effectiveness.

Tactical-Level Security Metrics

Tactical-level security metrics provide visibility into the day-to-day happenings of their organization’s security operations. These metrics empower CISOs to optimize resource allocation at a team level and proactively address emerging risks. 

The tactical metrics for CISOs include: 

• MTTR: While this also applies at the operational level, at the tactical level, MTTR helps CISOs assess the effectiveness of their security operations teams in responding to incidents. Understanding the MTTR trendline can help CISOs answer questions like “Are attacks increasing in sophistication?” and “Is there a process improvement or investment that will help decrease MTTR?” By monitoring and reducing MTTR, CISOs can ensure that their teams are equipped to handle security events promptly and minimize the potential impact on the organization.
• Event count and true-positive count: Event count and true-positive count provide insights into the volume and quality of alerts being generated. Evaluating the true-positive count can help assess the efficiency of alert tuning and identify the need for further refining detection capabilities.
• Criticalities: Analyzing the criticality of alerts is essential for identifying potential risks. If all alerts being worked on are of low criticality, there may be an opportunity to automate, or it could indicate the need for further tuning.

The most critical use of tactical-level metrics is to glean insights about what’s going on in your environment. For example, if an analyst on one shift sees a significant increase in alerts compared to the previous shift, it could indicate an ongoing attack or the need for tuning newly implemented security controls. By closely monitoring these shifts, CISOs can proactively identify potential problems and take swift action to mitigate risks.

Conclusion

By utilizing a comprehensive range of security metrics across strategic, operational, and tactical levels, CISOs can gain a holistic understanding of their security program. These metrics provide valuable insights, enabling them to make data-driven decisions, prioritize actions, and drive continuous improvement. Ultimately, the effective use of security metrics empowers CISOs to build robust security programs that align with business objectives, mitigate risks, and protect the organization from cyber threats. 

Learn More 

No matter where you are in your security operations journey, ReliaQuest can help guide you in the right direction. To learn about how GreyMatter and the Security Model Index can help you overcome security challenges and improve performance, reach out to request a demo.