Security Event Logging and Monitoring Techniques for Incident Response in Hadoop

This presentation will share some of the techniques and lessons learned in real-world Hadoop implementation at Johns Hopkins. Data will be sanitized as expected. But the focus will be on strategies and techniques used to collect and monitor audit and access log events from key Hadoop services and forwarding to a central server for monitoring, analysis, and response to any suspected breaches or incidents. Automation techniques, such as Ansible scripts to install agents or forwarders uniformly and efficiently across the cluster nodes will also be highlighted where appropriate.