A Journey of Audit Logs from Kafka to Elasticsearch - Building a Centralized Logging Platform

A Journey of Audit Logs from Kafka to Elasticsearch - Building a Centralized Logging Platform

Explore a 26-minute conference talk from Data Science Conference Europe 2023 that delves into building a centralized audit logging platform for tracking data modifications across multiple users and systems. Learn how managed cloud services like Kafka (MSK) and OpenSearch (Elasticsearch) can be leveraged to create a scalable solution for handling audit logs in business-critical data environments. Through architecture explanations and live demonstrations, discover how to implement and search audit log services effectively, gaining insights into maintaining data accountability and traceability in complex systems.