Scaling Climate Data for FinTech with an Open Source Data Mesh
Explore the Open Source Climate (OS-Climate) community's innovative approach to building a scalable open data platform for climate-related financial technology in this 35-minute conference talk. Learn how OS-Climate tackles the challenges of ingesting, processing, and managing quality data from diverse sources such as corporate climate reports, business entity graphs, and climate impact models. Discover the implementation of a fully open data mesh using open-source software like Trino, Open Metadata, Airflow, DBT, and Open Data Hub. Understand the importance of scaling not only compute and data but also application deployments, data pipelines, and human-searchable metadata access. Gain insights into developing data products using data-as-code principles and leveraging data federation architectures to minimize redundancies. Get inspired to contribute to the OS-Climate community and learn how to build a modern data mesh using open-source software and open data principles.