Analytical Tradecraft at Digital Shadows (now ReliaQuest)

This week my colleague and I attended the SANS Cyber Threat Intelligence conference in Washington DC. It was great to hear more from analysts and CTI users from across the community, as well as mingle with the plethora of vendors who were present. One of the talks that really resonated was by Forrester’s Rick Holland, entitled ‘State of Cyber Threat Intelligence Address’. Rick emphasised how analytical tradecraft is critical to operational and strategic CTI solutions. It was good to see Rick highlight the classic book by Richard Heuer called ‘Psychology of Intelligence Analysis’. Heuer was an analyst with the CIA and a huge advocate of Structured Analytical Techniques (SATs).  These give intelligence analysts structured frameworks for breaking down analytical problems into tractable components. Here at Digital Shadows (now ReliaQuest) we use some of Heuer’s techniques, in particular Analysis of Competing Hypothesis (ACH). Using ACH our analysts can record their assumptions, evidence and hypotheses on one matrix. Once added and scored they indicate which hypothesis the evidence suggests is least likely to be valid. (ACH doesn’t seek to prove hypotheses; it instead disproves them). By using tools such as these our analysts can structure their thinking and assessments in one place, allowing easier collaboration and peer review by colleagues.

Another useful by-product of ACH and its structured techniques is that they often offer transparency in assessments. There’s a great article from February 2013 by Jim Marchio in the Intelligence and National Security journal entitled ‘Analytic Tradecraft and the Intelligence Community: Enduring Value, Intermittent Emphasis’. In this article, Jim draws on research from the US Intelligence Community from the 1940s to 2013. Of particular note is the approach laid out by the US Intelligence Community’s Analytic Standards:

• properly describe the quality and reliability of underlying sources;
• properly caveat and express uncertainties or confidence in analytic judgments;
• properly distinguish between underlying intelligence and the assumptions and judgements of analysts;
• incorporate, where appropriate, alternative analysis;
• demonstrate relevance to US national security (for Digital Shadows (now ReliaQuest), this would be demonstrating relevance to our clients);
• use logical argumentation;
• exhibit consistency of analysis over time, or highlight changes and explain rationale;
• make accurate judgements and assessments.

For me, this epitomises what is required from intelligence analysts, and, when combined with structured analytical techniques, really enriches the quality of CTI. Here at Digital Shadows (now ReliaQuest) we ensure our analysis is objective, rigorous and transparent. But it’s not just structured techniques that achieve this: intelligence analysts also require awareness of cognitive bias and heuristic pitfalls and methods for reducing their effects. In the following blog posts we’ll be looking further into how cognitive bias manifests itself and what we can do to mitigate against it.