Awareness – Monitoring & Logs

Most attackers, or at least the clever ones with an aversion to orange jumpsuits, will want to cover their tracks.  They don’t want to be caught, they do want to keep their ill-gotten access, and they know that most logs are never reviewed.  Those that are reviewed are generally only reviewed after an attack has already become an incident.  Without complete and integrity protected records, security analysts are blind to probes, attacks, and the actions taken by attackers once the breach has been successfully executed.  The latest threats have become advanced and persistent.  They are custom built to avoid detection, by-pass security controls, exploit vulnerabilities and remain undetected for as long as possible, stealing information, credentials, and intellectual property.  Sometimes the only way to detect them is a review of audit logs after suspicious traffic patterns are noticed through monitoring.

Audit log records are evidence, and regardless of whether or not you are investigating a breach, should be treated as such.  They are evidence of both normal, secure status, and anomalous, suspicious activities.  They can provide times, dates, list commands executed, objects manipulated, errors returned, and success/failure details that can aid in the construction of timelines, and when corelated, indicate entry point, attack source and destination of any stolen data.  Regulatory compliance requirements may dictate that a minimum level of logging be performed, and logs kept for a time and/or reviewed.  Poor or no log analysis processes can lead to undetected system compromises for months or even years.  Can you spell WikiLeaks?  Could your personal reputation and your company’s reputation withstand a major breach and subsequent malicious disclosure?

If you ever have to prove in court that your company did what it was obligated to do in some legal context, or even worse, DIDN’T do something that it was REQUIRED NOT to do by law, (like attack another network or allow inappropriate access) you are going to need some very detailed logs, or plenty of money, guns and lawyers (shout out to Richard G!).

  • Systems should record logs in a standardized format such as syslog or those outlined by the Common Event Expression initiative.
  • If systems cannot generate logs in a standardized format, log normalization tools can be deployed to convert logs into a standardized format.
  • Validate audit log settings for each device and the software installed on it, ensuring that logs include date, timestamp, source and destination addresses, and other appropriate elements of each packet or transaction.
  • Ensure that all systems that store logs have adequate storage space for the logs being generated.
  • Logs must be archived and digitally signed on a periodic basis.  The sooner the better.
  • Real time log archiving to a central store with replication and backup is best practice.
  • All remote access to the DMZ or the internal network should be logged verbosely.
  • Operating systems should log failed access control attempts to all resources.  Successes too in some cases.
  • Security staff and system admins should run regular reports to identify anomalies in the logs, investigate any anomalies, and document their findings.
  • Include at least two synchronized time sources for all network equipment to retrieve time information from so that timestamps remain consistent.
  • Network boundary devices should be configured to verbosely log all traffic allowed and blocked at the device.
  • For all servers, ensure that logs are written to write-only devices or to dedicated logging servers on separate machines from hosts generating the event logs.  This reduces the opportunity for an attacker to manipulate log files.
  • Use SEIM tools to create baselines of what is considered “normal traffic” patterns and volumes.
  • Profile common events from each system to tune detection on unusual activity, reduce false positives, improve anomaliy identification, and reduce unnecessary alerts.

Basically, if you can’t see what is happening, and what has happened, on your network, it really isn’t YOUR network.  If you can’t prove that everything is normal and working as expected, it probably ISN’T.  Log your network’s activities, and review those logs quickly and often.