Reviewing Equipment System Logs - Do I have to?
Pete Boergermann - BankInfoSecurity.com Contributor
Gone are the days when we could just throw a hub on a closet shelf, run a few network cables, connect some PCs and a server to it and have a network. Logs? What logs? Why would we want to look at them? Times have changed and most devices connected to your network have logging capabilities. These devices have the ability to produce large amounts of valuable data. But it can be overwhelming to manage. A new industry that creates technology to manage security event logs is just starting up. As this technology matures, we may end up with products that can correlate the data between devices and alert us to events on a global multi-device level. Maybe these new products will be able to learn and adapt to new event information, possibly make assessments based on trends, then send only the alerts that need to be acted upon. Now that securing of our networks is so important we should be asking questions like: "What do we log, and why?" "How often do we need to look at it and who should review them?" Then reality hits and these comments come to mind... "I really have other things I need to do" "Reviewing them is boring and time consuming." "I will get to them tomorrow."
Let us start by defining what system log data is. Most network-able equipment has the ability to record, based on predetermined settings, events or a history of activity. These events, called system logs, can be as simple as a port on a switch getting disconnected or as specific as a table on a database running out of room. The format of the information is typically defined in the following categories: Critical, Warning and Informational. They may also have a minor and major rating. The information contained in the events can be easy to read or very cryptic and each manufacturer seems to follow a different standard. So, some equipment system logs can be a great source of information and others are completely useless. To get to these logs, you will need to connect and log into each device individually to view the information. Be warned, unless you have a centralized repository to collect the data, a considerable amount of time can be spent getting to and reviewing that data.
So where do you start? First, define a risk profile for your network equipment. Which devices are critical to your network infrastructure? A switch with PCs and printers connected to it would be low risk. A switch with servers attached to it would be a high risk. The reasoning behind this is, if a couple of PCs get disconnected it would only affect a few users; if a server gets disconnected it could affect all your users. Firewalls, Routers, and Servers are network devices that should have logging enabled without question. Next review your logging options with the Vendor provided documentation. What are the recommended settings? Debug mode logging can cripple your device, rendering it useless; verbose mode may only need to be used when you are experiencing problems. If you are experiencing unusual or unexplained problems, these system logs will prove very valuable. Spend some time reviewing them. SANS Institute teaches a very valuable concept, "Know Thy Network". If you know what typical behavior is, then non-typical behavior is easier to detect. This is best done when there's not an emergency.
Now for the question that makes most network professionals squirm before answering: How often should you look at this information? Initially we need to discuss Regulations. What industry are you in? If you are a member of the Financial or Health Care industry, then you need to know what is expected with reference to equipment logging before your next audit or examination. There are a number of regulations covering the financial industry, such as the Gramm-Leach -Bliley Act designed to safeguard customer information. Other regulations such as Sarbanes-Oxley, and the FTC Information Safeguards also require monitoring of your logs.
Here are what some of the Regulations say:
- "Review security procedures for daily and periodic report monitoring to identify unauthorized or unusual activities."
- "Determine whether logs are sufficient to affix accountability for host activities and to
- support intrusion forensics and IDS and are appropriately secured for a sufficient time period."
- "Appropriate logging and monitoring takes place.
- "...remote access logs can be reviewed daily for access during unusual times. Other logs can be reviewed on other regular cycles for other unusual behaviors...
- "Host Systems - Secure configuration (hardening) - Operating system access - Application access and configuration - Malicious code prevention - Logging - Monitoring and updating
Depending on the regulations you must follow for your organization and the risk profile you set for your equipment, some of your network devices may need to have their logs viewed daily. Talk to your auditors; they are not your enemy. Ask them what they will be looking for next time they visit you. If you can get a clear understanding of how they are going to audit IT Controls, you can save yourself a lot of grief and frustration. Automating this process as much as possible will save time.
There are some great tools available to collect system logs into a centralized location, but they are not cheep. They will, however, help your network team make sense of this overwhelming amount of information, plus most have built in alerting capabilities. Check out the following companies: LogLogic, Sunnyvale, Calif., TriGeo, Post Falls, Idaho, eIQnetworks, Acton, Mass., BindView, Huston, Texas and GFI, Cary, NC. Also take some time to check your regulations about archiving these logs. You may need to keep an untouched copy around for forensics, for a specific amount of time. Or they may need to be copied onto media that cannot be altered; in case they are requested for evidence of a break-in. Correctly complying to these regulations can be difficult, there seems to be a lot of room for subjective interpretation.
Finally, all these logs need to be reviewed by a competent network professional. Typically the person who installed the equipment is the best choice. That person usually understands the information in the system logs, knows how it should operate and is the one who needs to fix it when it's broken. Logbooks should be created for critical equipment and sharing the responsibility between multiple people helps. Log monitoring can get very boring! Do not forget your succession planning. If possible have a backup person who can interpret these logs.
The IT Manager should check up to make sure it is getting done. The last thing you want is to have the auditor show up to review the logbook only to see missing entries. Also, a documented plan of action for responding to security issues should be developed. Include a logbook of critical events that have happened and how you responded. This helps your employees know how to respond and whom to contact when something does happen. This logbook gives you documented proof that you are diligently watching and responding to security events. How your team responds should depend on the nature of the event. Expect some false positives as you start learning and keep in mind that high profile events may even fall under your Incident Response Plan.
Reviewing logs can help you be proactive in monitoring your network for equipment failures or could be used after a security breach to perform forensics. Translation: They can help you look like a hero. Monitoring them is a critical process in maintaining your network environment.
Tomorrow is here, and yes, you have to.
Pete Boergermann - A Master Certified Novell Engineer and a Cisco Certified Network Associate, Pete Boergermann currently holds the position of MIS Technical Support Manager and IT Security Officer at a mid-sized Community Bank in Pennsylvania. He actively serves on the Pennsylvania Bankers Association's Member Services Policy Committee. Pete has served on the Pennsylvania Bankers Association's Technology and Operations Committee for several years, chaired the committee, along with writing several articles for the PBA magazine. He has over ten years of experience in network development and implementation.