That's the punchline to an old physics joke about horse racing - it reflects the often-times unrealistic expectations we make when creating academic models for real-world performance.
I got thinking about this after http://www.bloginfosec.com/2007/07/18/email-from-dr-lawrence-gordon-security-roi-possible-but-not-optimal-use-other-metrics/">Ken emailed me about his blog post after reading my previous post on ROI. I think that http://www.bloginfosec.com/2007/07/18/email-from-dr-lawrence-gordon-security-roi-possible-but-not-optimal-use-other-metrics/
">his post definitely ends the ROI debate with some very smart (and diplomatic) comments from Larry Gordon.
More importantly, I spent some time with the Gordon-Loeb model for cyber-security investment after reading Ken's post, and it reminded me of the afforementioned joke. While it's an interesting paper from the perspective of provoking thought, I think there's a lot more to security investment than the model suggests. For example:
"The parameter Î» represents the monetary loss to the firm caused by a breach of security of the information set.... Even though we initially assume that this loss is a fixed value, we will investigate how changes in the value of the loss affect the firmâ€™s security investment decision."
This is where I get frustrated by a lot of infosec economic models (and why I was so simplistic in my own post) - we miss the point that information security does not only prevent loss, but (in most cases) has the side benefit of reducing operating risk. Think about it for a second... a vulnerability in a system is as much an issue of product quality as it is an issue of security vulnerability. (This can be discovered by a thought experiment: imagine a perfectly designed and perfectly implemented product with no defects - would vulnerabilities exist?)
In such, remediating the risk presented by security issues also reduces operating risk, leading to higher up-time, more environmental awareness, and better monitoring of system state. These aren't just loss-prevention activities, but actually lead to increased efficiencies and better effectiveness of technology.
I've yet to see a model take this into account - yet I see CISOs make decisions on that criteria (usually intuitively and without conscious understanding of why they're doing it) often.
Which is why I hate the whole argument from formal economic terms. The fundamental question is always a simple one:
How much does my business increase its net profit because I have purchased this technology/implemented this process/bought more toilet paper/hired this person/etc.?
Ask that question, and the debate about whether you call it ROI, IRR, Rate of Return, Cost Reduction, or any number of other things goes away.
And you're left with the only thing that really matters - a real horse that wins the race in the real world, not a spherical horse in a vacuum.