Entity Resolution at Scale

Entity Resolution at Scale

Explore entity resolution techniques for large-scale data cleaning in this 23-minute conference talk from YOW! 2019. Discover how Software Engineer Huon Wilson from CSIRO's Data61 approaches the challenge of connecting duplicate and corrupted records to their single underlying entity. Learn about the solutions and lessons gained from implementing entity resolution on Apache Spark to process billions of records. Gain insights into overcoming real-world data challenges, scaling data cleaning processes, and improving data quality for more effective analysis and decision-making.