The Center for Strategic and International Studies (CSIS) wants the commission to come up with a list of recommendations that the new president who takes office in January 2009 "can pick up and run with right away," said James Lewis, director of the CSIS Technology and Public Policy Program. The commission, made up of 32 cybersecurity experts, plans to finish its work by the end of 2008. I am fairly confident that nothing of value will come from this group, but there is one task which could completely reverse my opinion. Rather than wasting time on recommendations that will probably be ignored, how about taking a step in a direction that will have real impact: security metrics. That's right. Spend the first day (or two, if you are a slow reader or can't sit still for long periods) reading Andy Jaquith's book. Next, and this is the crucial part:
Figure out how to play and score the game before you pretend to think you can improve the score.
What does this mean? Just a few ideas include:
- Propose definitions for security, risk, threat, vulnerability, inside threat, external threat, and all the other words we use yet upon which we never agree. Hold hearings and invite real security people (not just digital security people) to express their views.
- Propose some metrics and see how other operations define success. Hold hearings on the results of that process.
- Apply metrics to some real organizations and gain a baseline set of numbers. Repeat the process at determined time intervals. Try to identify correlations and if possible causations. Be anonymous if necessary, but use a real methodology and not the self-selection applied by CSI/FBI and others.
Do you see where I am going here? At the end of the process we could have a framework for seeing just what is happening. I defy anyone to tell me just how bad or good our digital security situation is right now. Some say the sky is falling, others say we're happy! happy!, others say we're just as secure as we need to be to continue limping along. It is a proper role for a panel of worthies to help figure out how the game is played and then what the score is. It is a waste of time to make recommendations before those basic steps have been taken.