How Does Your Information Security Program Measure Up?

Developing a metrics measurement for your financial institution doesn’t have to be something that is dreaded or feared. Planning a metric program and implementing it to measure the effectiveness of your entire information security program can yield your group and your financial institution unseen benefits.

As Rebecca Herold, an information security and privacy expert noted during a recent interview, “Metrics are good. Not know how well your program is doing and improving isn’t.”

Herold, who is the author of “Managing an Information Security and Privacy Awareness and Training Program,” recommends multiple types of metrics. “You’ll need a variety of different ways to measure the effectiveness of your program,” she said.

The establishment of a baseline to figure out where your institution is at will help you effectively measure progress. Herold developed a metrics program for the information security department at the Principal Financial Group.

She said the measurements, once established, should be something, “You stick with, so choose something to measure that is sustainable over time.” Her metrics program, for example, would keep track of how many employees visited the group’s intranet webpage.

“When there was an announcement or change to policy or even a new awareness event, we’ d measure the number of ‘hits’ the new piece received, and after a while were able to track it to the number of attendees who came to an event,” she explained.

The variety of metrics you can use is innumerable, Herold said. One example she pointed to was the after hours walk through around the company. Information security group employees would “walk through” a department, noting deficiencies or policy violations such as written passwords, passwords taped to the bottom of key boards, or leaving an unsecured computer turned on. They also found more serious policy violations like leaving a negotiable check out in the open, rather than in a locked drawer.

“We would then take these reports back to the department’s management and let them deal with it,” she noted. They also did similar walk throughs before AND after information security awareness training, to measure the effectiveness of the training.

One other measurement that the information security group was able to get implemented was the inclusion of security compliance in the job review process. “Managers included it in an employee’s annual job appraisal process. They were asked how well the employee followed the information security policies, or if they were non-compliant,” she said.

In the training and information security awareness training measurement was essential. “Prior to the training we did a pre-questionnaire, to measure how much the person knew already about the information they were going to be presented.”

After that pre-training questionnaire, and the employee’s completion of the training, “we wanted to find out how much they knew, so we then followed up with a ‘non-quiz’ test after the training. We didn’t make them feel like they were being tested, but we wanted to measure what they knew about the subject,” Herold continued. The key point being whether they understood what was covered in the training.

Herold said the group would compare the after test score to the pre-training quiz that each employee was given to what level of improvement each one had after attending the training. “We would then go three months and six months afterward to retest them and see if they were able to retain that information covered in the training course,” she explained. These follow up tests showed if the training was effective, and it proved correct.

Evaluations and metrics can be a useful tool. “There are many different ways metrics can be worked into a financial institution’s information security program. Unfortunately many don’t even use the most basic ones, which in my estimation is a shame,” she noted.

One unexpected metric that Herold discovered was that the number of calls to the information security department increased from the areas that received the awareness training. “They knew who to call. And many times we could correlate the calls to a real event,” she noted. It can be a double-edged sword, “because we were spending a good deal of time answering and responding to calls.”

However Herold and her group took it in stride. “It was a good metric; it proved one thing, that the awareness training we provided was effective. If we didn’t get people calling after the awareness training, then the message wasn’t on target.”

About the Author

Linda McGlasson

Linda McGlasson

Managing Editor

Linda McGlasson is a seasoned writer and editor with 20 years of experience in writing for corporations, business publications and newspapers. She has worked in the Financial Services industry for more than 12 years. Most recently Linda headed information security awareness and training and the Computer Incident Response Team for Securities Industry Automation Corporation (SIAC), a subsidiary of the NYSE Group (NYX). As part of her role she developed infosec policy, developed new awareness testing and led the company's incident response team. In the last two years she's been involved with the Financial Services Information Sharing Analysis Center (FS-ISAC), editing its quarterly member newsletter and identifying speakers for member meetings.

Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing, you agree to our use of cookies.