Metrics Revolutions.
(Can I get Keanu Reeves over here to frown broodingly at a screen and say, “That’s odd ...“?)
Following on the previous post, Andrew Jaquith brings his alliterative and quantitative skills to the flat screen. How thrilling is this?
He makes a lot of sense with his post, but I’d like to pick apart some of his examples and try them on for size in MY world.
Likewise, for learning and growth, we want to spread responsibility for security, equip employees with the right security knowledge and skills, and promote adaptability in the face of changing threats. These themes imply the following typical objectives:
- Delegating responsibility for authoring user activities to business units
- Increasing collaboration between IT security and business units
- Ensuring effective levels of security certifications for security staff
- Promoting security awareness throughout the organization
- Integrating secure behaviors into employee’s everyday activities
- Ensuring that security features are easily understood and adopted
- Heightening awareness of emerging security threats
- Exploring discretionary security frontiers
- Giving employees the skills needed to properly handle security incidents
I’ve purposefully listed the objectives for both perspectives (internal process and learning/growth) as objectives, not metrics, because they illustrate the behaviors that organizations should be encouraging. Every one of them can be readily mapped to process metrics — key indicators — that show whether an organization is achieving those objectives.
Let’s try some process metrics for those then, shall we?
Delegating responsibility for authoring user activities to business units - I’m not completely sure I understand this, but okay, assuming you can measure their activity somehow, you can argue about how much they were doing as a result of having it delegated to them.
Increasing collaboration between IT security and business units - How do you measure this? By the number of business unit keggers that IT security gets invited to? By the decreasing number of flames being sent back and forth with a Cc: to senior management?
Ensuring effective levels of security certifications for security staff - Bwahahaha. Define “effective” and then we’ll talk.
Promoting security awareness throughout the organization - Okay, I do this now through the number of newsletters issued, the number of training classes provided and the number of attendees, and the results of an annual OCTAVE survey.
Integrating secure behaviors into employee’s everyday activities - What kind of secure behaviors are we talking about? Not going to MySpace? Not opening spam attachments? I suppose you could count the number of times screens were left unlocked, or if you were really ambitious and nosy, you could count the number of outgoing emails that should have been encrypted but weren’t. Other than that, though, I’m metric-less.
Heightening awareness of emerging security threats - How is this different from the awareness activity just above? Do we count the number of CERT advisories we mail around? Do we do another survey?
Exploring discretionary security frontiers - I think this means researching new security technologies, but I’m not sure. Okay, so we measure spending dollars and dedicated man-hours, or number of pilots resulting in new procurements, or something. Help me, Jean-Luc!
Giving employees the skills needed to properly handle security incidents - You mean, other than giving them a phone number to call? What are some metrics for this one?
Don’t get me wrong: I’m not arguing that these aren’t great objectives, or that we shouldn’t be focusing on security behaviors. Far from it. I’m just having trouble making the leap from objective to mapped metrics. Then again, Jaquith is doubtless much smarter than I am. I am Only An Egg.
Posted by shrdlu on Friday, November 17, 2006
(5) Comments • Permalink •
