Josh Richards "jrichard"

I have a love/hate relationship with audits.

Let's talk about them in the IT realm. I think they have the potential to do some real good. And, in certain cases, they have. Just how much is very difficult to quantify with any degree of confidence.

We could look for trend changes, for a variety of problems targeted by audits but.... it's not easy to prove a cause and effect relationship. No doubt marketers of auditing and related consulting services can come up with some statistics but I don't buy it.

For example, even without audits, it's reasonable to expect a variety of problems to plateau as adoption cycles peak. The acceptance and understanding of particular problems -- and their solutions -- improves as time passes. Heck, sometimes having one really widely publicized (and costly) hit on another organization can be more effective than an audit at getting problems fixed (at least in the short-term). There are loads of other possible factors, including macro & micro-economic.

So, for the moment, I'll go with that I do know about the impact of audits -- from actual observation.

Audits certainly aren't "fun" so why do we care about all this? Well, because if we're going to have to go through the process anyway -- and probably repeatedly -- why not try to get as much benefit as possible out of it? Think of it as trying to get the most bang for your buck...even if you were forced to spend the buck. :-)

Many audits I've seen suffer from some variation of the following flaws (I'll discuss the positives momentarily):

• They encourage organizations to overlook the forest for the trees. Management is happy because the auditor left without an adverse opinion. IT is happy because people are off their back. Life goes on and, especially in environments where firefighting is the norm, things get forgotten that weren't truly covered in the audit. For example: should we be doing anything with the data from that brand spanking new intrusion prevention system purchased, initially, for the audit? (sometimes it's left there until the next audit anniversary comes around when they get to point it out to the auditor with a look of relief...and hope the auditor doesn't want to strike up any small talk about "what they think of it" thus far).
• They increase the burden on the IT department. While a super on the ball IT department without any extra resources might be hurt by this, for most others this may be a wash since it'll catch a few things they might have missed and help them justify some much needed resources.
• They create a false sense of security for management, staff, users, and customers (kind of like airport security inspections)
  
• Audits are not created equal and they are difficult to compare quantitatively. Most of the intended beneficiaries (end-users, customers, clients) of audits are certainly not equipped to be able to tell any two audits apart or evaluate the actual versus perceived merits of a given audit.. Hacker Safe(tm) anyone?
• The more useful audits tend to be the most pain-in-the-ass. Thus no one wants to undertake them (except the auditor who is likely charging significant fees). Ironically the least useful fall into BOTH camps: some are ridiculously painless (and useless) on a technical basis while others are a severe pain-in-the-ass with minimal, if any, benefit.
  
• Most widely used (and publicized) standardized audits are either too rigid or too vague. It's tough to get both widespread adoption of a standard spec (e.g. PCI) that is both highly specific and relevant. While there are some best practices, there are no two environments that are exactly the same. And, hell, who is to say that the owner's decision to simply buy really good liability insurance is imprudent?
• Human nature gets the better of us. No one likes being told what to do and explaining our (sometimes sloppy and other times simply pragmatic) past decisions is rarely fun. An adversarial feeling and (at least a bit of) resentment is not uncommon. This leads to sliding things under the rug. Often the IT folks already know the scary areas. Too bad there isn't a good way for them to disclose all that information..... for the constructive benefit of the organization...
  

On the positive side:

• Audits have provided fuel to IT managers and staffers in justifying investments in infrastructure enhancements. In competent hands, and as long as nobody goes overboard, this is good stuff.
• They occasionally catch stuff missed even by the most competent.
• Seriously negligent organizations are (more likely to be) caught sooner (or at least their cost of hiding is raised enough that it becomes more cost effective for them to simply get up to par rather than fight).
• Audits get management attention. Very productive discussions about security, risk, and policy management often come out of this.

Which of these same situations may have occurred in your own organization?

By starting some discussions within your organization -- not just within the IT department, but also within the executive suite, and perhaps in consultation with a constructive representative on the legal side -- resources consumed by the audit process can made more effective.

While no on likes to spend time & money they didn't chose to, at least you still have control over the specifics of how it is spent and what the long-term benefits can be to the organization. It's got to be a better result than simply passing the latest audit.

(As an aside, an idea I've been kicking around with some of my clients is aligning the marketing folks with the IT department to publicize positive audit results and to justify more extensive investments, that still make sense in a business sense, beyond simply what the audit minimums are. Thus creating a potential competitive advantage out of the whole thing. An idea to kick around in your own organization perhaps).

-jr