The Storage Architecture of Intel's Data Management Platform
Explore the storage architecture of Intel's Data Management Platform (DMP) in this 27-minute conference talk from USENIX Vault '19. Delve into the rack-centric cluster design utilizing an Ethernet-based fabric interconnect and a unique approach to storage redundancy. Learn about the DMP's overview, including its Kubernetes management and use of Container Network and Storage Interface providers. Discover the two volume types: NVMe-over-Fabric based Remote Logical Volumes and Local Logical Volumes optimized for Optane DC Persistent Memory. Gain insights into workload priorities for initial DMP deployments, including a Spark-based AI/Analytics Pipeline and MySQL/MariaDB transactional database on shared storage. Conclude with an update on the DMP effort, preliminary performance results, and future steps for this innovative storage architecture.