Rating your pentester.
Dan Morrill had a great blog entry a while back about the difficulty in finding good third-party consultants to do penetration testing. I always meant to comment more on that, but never got around to it. Then Tate Hansen at ClearNet Security came out with a great flowchart basically illustrating the same thing: that a good pentester doesn’t just fire up the tools and send you the results.
The best analysts I have worked with have (1) taken time to understand the business; (2) inspected the network (when not on a black box engagement); and (3) put processes together with vulnerabilities and leaked information to discover combinations that led to compromise. They used Google hacking, mined information from our websites, and manipulated system parameters and application behavior to break in. (In other words, they worked like a REAL hacker would, not like a script kiddie who knew how to run Nessus.) They considered human foibles in procedures and processes as well as coding weaknesses. I got a holistic view of our exposure instead of just a scan report that said, “You need to turn off these ports and strengthen your passwords.“
When you’re having a third-party assessment done, you really need analysis, not tool-using. Some things that I like to be told are how my organization stacks up security-wise against similar organizations, and what fixes we can make that will bring the biggest, most cost-effective improvement in our posture. The best consultants I know are businesspeople and system architects as well as security professionals.
If anyone’s shopping around for consultants, drop me a note here and I’ll give you a list of my favorites. 
(1) Comments • Permalink •
