Keeping up with the dot-Joneses.
Today’s rumination is brought to you courtesy of a metrics thought process gone out of control. We’re going to talk about just one topic, which is:
How do you compare your organization to someone else’s for the purpose of getting security metrics?
I see that executive sneaking in at the back, trying to get a ROSI number without anyone noticing. He wants to know whether he’s allocating his security resources as well as the CIO next door. Fine, let’s see what he’s trying to ask, and let’s dissect the questions.
- How do we find a comparable set of organizations to compare ourselves with?
We’ll throw a few stats at the wall and see how well they stick. For each one, we’ll ask:
1. Does this statistic tell us the same thing when we apply it to different organizations?
2. If not, what else do we need to know to make this number comparable and meaningful?
3. Why do we think this statistic will tell us something useful about security resource allocation?
Number of employees.
1. Do 500 employees here equal 500 employees there?
2. Maybe we think that 500 employees within the same industry are more comparable? If so, how?
3. Do we think that the number of employees correlates somehow with the difficulty of ensuring security?
Number of IT employees versus regular employees.
1. Is there some magic ratio that makes our organizations comparable if they match?
2. Does it matter what those IT employees are doing versus what the non-IT employees are doing? Again, does industry matter?
3. Does a higher ratio make security more or less challenging, or is it a non-sequitur?
Number of dedicated IT security employees.
1. Again, is there a magic ratio that works everywhere?
2. Does it matter what the IT security employees are doing, or does sheer headcount make a difference?
3. We’re assuming here that “dedicated” == “better at security.” Probably a safe assumption, but you never can tell.
Number of applications supported by the organization.
1. Does it matter whether the applications are being sold or just used internally?
2. What other information would we need to know about the applications to be able to compare real levels of security difficulty? How about number of legacy applications, funding for development and remediation, and whether the applications are tied to revenue generation?
3. The assumption is that the more applications you have, the more complex the environment you need to secure.
Number of networked hosts.
1. A host is a host from coast to coast?*
2. How about heterogeneity of hosts? Or number of sites (which may imply system and organizational complexity)?
3. This is a pretty straightforward assumption of security complexity right here.
Overall market capitalization.
1. Is capitalization the same across the world?
2. Does knowing the type of industry tell us more about the assets to be secured?
3. Does this say anything about the amount of money you should have available to spend on security? Or anything about what you should be spending on security to protect the assets represented by this capitalization?
IT budget.
1. Again, is an IT budget dollar the same at every organization?
2. Does it make a big difference what the IT dollars are being spent on?
3. Are we going to use the tired old “security budget as a percentage of IT budget” rule? What does this really tell us?
Are there some other/better ways to compare organizations in order to see how your security investment stacks up? And in the end, what do you really care what the dot.joneses are doing?
*A host is a host from coast to coast
And no one will talk to a host that’s close,
Unless the host (that isn’t close)
Is busy, hung or dead.
