Data Indexing with RGB: Ruby, Graphs, and Bitmaps - Lecture
Explore the journey of building a unique data engine using Ruby, a graph database, and a bitmap store in this RubyConf 2022 talk. Dive into Zappi's data history, starting with the challenge of disconnected data sets and serialized data frames, and discover how an in-memory index solution was developed. Learn about utilizing RedisGraph to model data relationships and connecting semantically equal nodes. Examine the implementation of a query layer for indexing a bitmap store, resulting in significantly faster dataset interrogation. Follow the evolution from the world before MeasureStore to the introduction of cross-market analysis, harmonization, and the integration of bitmap stores and graphs. Gain insights into the architecture, performance improvements, and future steps for this innovative data indexing approach.