How to Do Application Logging Right

Anton Chuvakin (the Security Warrior himself) and I have a paper in the current IEEE Security & Privacy Journal - "How to Do Application Logging Right." The paper explores app logging from a developer's perspective. There are various standards that mandate logging, most famously PCI DSS. PCI mandates a regime around log storage and security and provides an event model and log format for certain data (like financial data). For developers though its does not provide guidance on audit logger placement, what's useful to include and not include in event payloads. 

In my experience, Audit logs are one of the quick, dirty and cheap things that can improve enterprise security. Quick, dirty and cheap are a very rare trifecta in enterprise security and that by itself makes it worth paying attention to, but there are other good reasons for building visibility into your applications, that's neatly described by G.K. Chesterton (emphasis added):

The real trouble with this world of ours is not that it is an unreasonable world, nor even that it is a reasonable one. The commonest kind of trouble is that it is nearly reasonable, but not quite. Life is not an illogicality; yet it is a trap for logicians. It looks just a little more mathematical and regular than it is; its exactitude is obvious, but its inexactitude is hidden; its wildness lies in wait.

Access control models implement authentication and authorization models that rely on accurately identifying the subjects, objects, rules, conditions and actions that must be present to make an access control decision. This is sufficient to mitigate many threats, but does not account for all and specifically does little to address intentional misuse. This is where the accountability layer from such tools as audit log observers is essential. Monitoring has been confined to networks, which lack context that is available in the app and data layers, now we are entering an a moment where this is becoming apparent and starting to see the some large organizations putting monitoring at the app and data layers to understand the use and misuse of those resources.

There are several things an app can do more effectively than any other part of the stack. First you can add event-specific triggers to initiate some action, next the app has access to additional context that it can pull in as necessary, such as session data. Take an example like a web app that is reporting on vanilla HTTP information, a logger at the web server level just reports on request and response strings, however an app logger could have a trigger for a sensitive event (say, transferring money out of an account). In this case the logger can dump all session variables to the audit log including the authentication, access events, record management, transaction data, and other useful information.

 

So not only does the app have context that is not available elsewhere it can also be used to gather that additional context from sources.

Comments

[John Melton]

Just wanted to point out ESAPI has thought about logging and event/intrusion detection - http://www.owasp.org/index.php/Category:OWASP_Enterprise_Security_API, and the AppSensor project is intended to provide more extensive event/intrusion detection at the application layer that can be a drop-in replacement for the default ESAPI version - http://www.owasp.org/index.php/Category:OWASP_AppSensor_Project

[Clerkendweller]

I have most often used the EASPI Javadocs as the best guide for application security logging. But this new paper is very helpful. Perhaps the paper doesn't include the concept above of logging extended information about certain events. For example, we may not have a username, only an IP address/client fingerprint. And the request headers and/or response content may be helpful for subsequent analysis.

Writing application logging requirements into specifications can lead to more security being built in: if you need to log it (and thus deliver software that meets the spec), you have to be able to identify the event occurring.

John's mention of OWASP AppSensor is also timely as I'm trying to work up a logging/event syndication format for AppSensor events.