How Does Your Information Security Program Measure Up?
Developing a metrics measurement for your financial institution doesnâ€™t have to be something that is dreaded or feared. Planning a metric program and implementing it to measure the effectiveness of your entire information security program can yield your group and your financial institution unseen benefits.
As Rebecca Herold, an information security and privacy expert noted during a recent interview, â€œMetrics are good. Not know how well your program is doing and improving isnâ€™t.â€
Herold, who is the author of â€œManaging an Information Security and Privacy Awareness and Training Program,â€ recommends multiple types of metrics. â€œYouâ€™ll need a variety of different ways to measure the effectiveness of your program,â€ she said.
The establishment of a baseline to figure out where your institution is at will help you effectively measure progress. Herold developed a metrics program for the information security department at the Principal Financial Group.
She said the measurements, once established, should be something, â€œYou stick with, so choose something to measure that is sustainable over time.â€ Her metrics program, for example, would keep track of how many employees visited the groupâ€™s intranet webpage.
â€œWhen there was an announcement or change to policy or even a new awareness event, weâ€™ d measure the number of â€˜hitsâ€™ the new piece received, and after a while were able to track it to the number of attendees who came to an event,â€ she explained.
The variety of metrics you can use is innumerable, Herold said. One example she pointed to was the after hours walk through around the company. Information security group employees would â€œwalk throughâ€ a department, noting deficiencies or policy violations such as written passwords, passwords taped to the bottom of key boards, or leaving an unsecured computer turned on. They also found more serious policy violations like leaving a negotiable check out in the open, rather than in a locked drawer.
â€œWe would then take these reports back to the departmentâ€™s management and let them deal with it,â€ she noted. They also did similar walk throughs before AND after information security awareness training, to measure the effectiveness of the training.
One other measurement that the information security group was able to get implemented was the inclusion of security compliance in the job review process. â€œManagers included it in an employeeâ€™s annual job appraisal process. They were asked how well the employee followed the information security policies, or if they were non-compliant,â€ she said.
In the training and information security awareness training measurement was essential. â€œPrior to the training we did a pre-questionnaire, to measure how much the person knew already about the information they were going to be presented.â€
After that pre-training questionnaire, and the employeeâ€™s completion of the training, â€œwe wanted to find out how much they knew, so we then followed up with a â€˜non-quizâ€™ test after the training. We didnâ€™t make them feel like they were being tested, but we wanted to measure what they knew about the subject,â€ Herold continued. The key point being whether they understood what was covered in the training.
Herold said the group would compare the after test score to the pre-training quiz that each employee was given to what level of improvement each one had after attending the training. â€œWe would then go three months and six months afterward to retest them and see if they were able to retain that information covered in the training course,â€ she explained. These follow up tests showed if the training was effective, and it proved correct.
Evaluations and metrics can be a useful tool. â€œThere are many different ways metrics can be worked into a financial institutionâ€™s information security program. Unfortunately many donâ€™t even use the most basic ones, which in my estimation is a shame,â€ she noted.
One unexpected metric that Herold discovered was that the number of calls to the information security department increased from the areas that received the awareness training. â€œThey knew who to call. And many times we could correlate the calls to a real event,â€ she noted. It can be a double-edged sword, â€œbecause we were spending a good deal of time answering and responding to calls.â€
However Herold and her group took it in stride. â€œIt was a good metric; it proved one thing, that the awareness training we provided was effective. If we didnâ€™t get people calling after the awareness training, then the message wasnâ€™t on target.â€