The Naked Analyst

By Roger Davies / 21 February 2013 / Intel Analysis, Intelligence 101

IMSL have been doing some interesting work on examining “risk” in terms of  calculating probabilities.  We’ll make you aware of this interesting work in due course, but it’s pretty ground breaking.  And it’s made me re-look at some commonly held beliefs and concepts within the security and risk management industry.  I’ve revisited a number of commonly presented “equations” with regard to risk, and come to the startling if somewhat provocative conclusion that they are almost universally garbage.

This is slightly tricky since a couple of IMSL analysts (by coincidence) have quoted some of the equations in recent  (excellent) posts.  But I think that’s indicative of how pervasive the risk equations have become in the security and intelligence industry and I’d like, politely, to challenge the rationale behind the equations.  Because I think they are wrong.

Let me explain.  I come from a scientific/engineering background. Drilled into every scientist and engineer is the importance of defining units. So, for example, the equation for calculating Kinetic Energy is one half x the mass, x the velocity x velocity.  In nice metric units one can calculate the Kinetic energy of a moving object, Easy.  The units for KE are joules if the mass is in Kg and the velocity in meters per second.

The circumference of a circle is 2Pi x radius. The unit for the circumference is distance, and the radius is a distance so multiplying the radius by a quotient (2 x Pi) we get another distance in the same units.

I could go on.

But let’s look at some commonly quoted risk equations. I’ve seen these presented by numerous “security managers” and risk management consultants:

Risk = Threat x Vulnerability.

Really?  And what makes you think that that’s a valid equation? It sounds sort of right initially and there are some instinctive logic in the words at first. But the more you think about it the more it is nonsense.

  • What are the units? (There are none)
  • Has anyone ever defined what “units” Risk is measured in? (No)
  • Has anyone ever defined what “units” Threat is measured in? (No)
  • Has anyone ever defined what “units” Vulnerability is measured in? (No)
  • Have you ever seen anyone perform an actual calculation using this equation? (No)

So what is the point in giving it an equation?  There is none. It’s meaningless.  Then,  as you research the security and risk management industry you’ll come across this equation

Risk = Threat x Vulnerbaility x Consequence.

So there’s another ill defined item,  consequence, with no known units by which you are multiplying your already vague risk quotient.   Here’s another equation:

Asset + Threat + Vulnerability = Risk.

At the risk of being rude, I think this is trite nonsense.  Again, no units defined. How would one “add” a measure of Threat to a measure of Vulnerability – are they the same unit?  Not only that but this equation means that you can have an asset with no threat and no vulnerability and it still has a risk.

Here’s another I found today:

Threat x Probability x Business Impact = Risk.

This throws in that interesting concept “probability”… but no-one who ever uses this formula ever will define probability as an exact number, yet.

Lets look more deeply at these words: Risk, Threat and Vulnerability.  They are entirely subjective and ambiguous. They are largely ill defined and there are many definitions that exist. Risk, is only a qualitative, comparative (maybe even instinctive) feeling about the probability of something happening. It cannot be derived from an equation with no units.   Not only that, but these equations fail to deal with multiple security threats.

So next time you hear a security consultant stand up and make a presentation that uses on or other or a variant of these equations – ask him to work it through for the asset he is advising you on.  He or she will be flummoxed. It’s just a powerpoint slide filler, for people to nod sagely at. The Emperor has no clothes.

Bananas = washing up liquid x the offside rule plus the Daily Mail makes as much sense.

So without throwing more rocks, and be totally negative, let me suggest an alternative developed by Jeff Lowder:

Risk = A function of (Threats, Vulnerabilities, etc) 

Or

Risk = f(Threats,Vulnerability,etc)

I think that’s more logical, and suggests, correctly, that the complexity of a risk is affected by the Threats, the Vulnerabilities and other things. And it doesn’t have or need units, because the function is undefined.  That “function” is important, and frankly that the job of the intelligence analyst to work out for every single case, each time.  So the importance to IMSL of this whole argument is that by analysis we define the “function” of threat and vulnerabilities and other issues which when brought together helps understand the risk. Risk is something to understand not calculate in absolute terms.

In broader terms, and in English not Maths:

Intelligence Analysis of Threats will define the Risk. Other factors such a Vulnerabilities and Impacts will also be taken into consideration.

If anyone tells you they can “calculate” the risk using an equation they are bullsh*tting, and you can prove this by asking them to describe what units risk is measured in, As my old maths teacher used to say  “show your working”.

One comment on The Naked Analyst

  1. Muso

    I was always taught that if in doubt assume the units to be Coconuts per Fortnight.

    Such a fabulously practical system of units is clearly applicable to the rate at which you may receive a bruised head sitting under a tree on a desert island – itself an inherently risky business. 🙂

    Reply

Leave a Reply

Your email address will not be published.