There is an excellent article in this month’s “Nature” journal about the way in which scientists fall foul of cognitive biases in the analysis of their scientific data. This matter has an exact read-across into intelligence analysis where the same cognitive biases of course apply. The article provides some key techniques for countering these cognitive biases, which I’m glad to say are pretty close to those measures taught at the IMSL Intelligence Training School within some of our courses.
What should worry us as Intelligence Analysts is that the scientific world has a process of peer review from experts outside of any one institution to carefully examine scientific reports and the focus of these peers should always in part focus on identifying and querying such biases before publication. Despite this, I think it is true to say that bias in scientific reports is endemic. In the Intelligence world external peer review rarely, if ever, happens, perhaps for understandable reasons. But that further increases our liability for bias and for adopting group-think within any one intelligence organisation. Even more so, the opportunity for an analyst to hide behind “need to know” principles and the necessary but arcane security classifications of sources, further exacerbates the ability and perhaps the inclination of intelligence analysts to hide the reasons for their derived assessments. The scary truth is that no-one knows how much cognitive bias is affecting intelligence outputs within any organisation. That really should worry us. If you are an intelligence analyst take a look at how the a “just so” story bias occurs in the article mentioned above. I’ll bet a lot of beer tokens that you will have encountered that in your duties. Did anyone ever tell you how to avoid that when you went through training? Were such biases even discussed in your department? I doubt it.
I think that there are some answers though, which although they won’t solve the problem completely at least begin to address the issues. Some of the techniques that are being applied in the scientific community are covered in the Nature article, and have application directly into most intelligence departments or agencies. You will note too that the article uses as an example a cognitive bias that seriously affects criminal investigations with regards to statistics.
The IMSL Intelligence training school is considering launching a specific and focused training course in such techniques – let us know if you are interested.