We’re all swimming in data. There’s data everywhere. From packet captures to reputation feeds, it feels like there is a fire hydrant of data flooding analysts. But are there any risks with this approach? One challenge is that the more hay you make, the harder it is to find the needle. Luckily we can employ analysts to sift through the data, and equip them with data mining and investigation tools to make sense of the noise. This approach is advocated across information security, and there’s a plethora of vendors providing solutions to this problem.

However, an often cited problem with this approach is that analysts can be overwhelmed by the volume and frequency of the data available. This leads to what Rick Holland describes as “indicators of exhaustion,” whereby we are overcome and almost paralysed by the onslaught of data.

But what about the data you don’t have access to? The data you don’t even know exists? Or to quote Donald Rumsfeld (sorry!), your Known Unknowns, and Unknown Unknowns. An overeager focus on “indicators of exhaustion” may overshadow the fact that you’re not even collecting the right data in the first place, and lead to an illusion that your main concern is triage and analysis, and not MORE collection.

This issue of Known Unknowns and Unknown Unknowns is commonplace in the intelligence world, and is often referred to as “intelligence gaps.” An example of an intelligence gap would be your organization being targeted with a DDoS extortion attack, however you know nothing of the credibility of the attacker.

Obviously Unknown Unknowns are harder to identify, and it’s pretty tough to figure out what you don’t know that you don’t know. The risk with these is that you have intelligence gaps that you’re ignorant to. For instance, there maybe a new malware campaign targeting your sector, but you don’t know it exists. Therefore, you don’t know its capabilities, nor how you can defend against it.

So what can you do to help? You need to understand, expect, and prepare for intelligence gaps. The worst thing an organization can do is to delude itself that its intelligence collection is complete and undefeatable. At Digital Shadows (now ReliaQuest), we constantly review the threat landscape to ensure we are collecting effectively against the threats which affect our clients, and that we are horizon scanning for ones which are developing.

When we identify a threat which we are not collecting against, we prioritize this against others, begin collecting against it, and also investigate why we had not discovered it sooner. Similarly, if a known threat suddenly surges in activity that we did not expect, we reprioritize our coverage of it. This ensures that intelligence gaps are kept to a minimum.

As for Unknown Unknowns, we hold regular reviews of our collection plan and understanding of the threat landscape. By conducting this in a collaborative and critical setting, we can maximise our collective understanding of the threats, and test developing hypothesis for relevance and likelihood. Therefore converting Unknown Unknowns into Known Unknowns, and slotting these into our collection plan for remedying.

Together, these two approaches reduce the likelihood of intelligence gaps and, more important, help to ensure that our clients reduce their own intelligence gaps.