Analyzing Configuration of Cellular Networks at Nokia with Apache Hadoop, Apache Spark, and Presto SQL
Explore a comprehensive conference talk on solving Big Data challenges in cellular network configuration analysis at Nokia. Dive into the implementation of a massively parallel processing, data warehousing, and visualization system using Presto SQL and various Apache Hadoop components. Learn about the project's approach to handling complex network configuration data, including size, inflow, complexity, and skew. Discover the software components utilized in the data pipeline, such as Cloudera CDH, MapReduce, Spark, HIVE, Impala, and HBase. Gain insights into the pitfalls encountered during development and operation, including performance issues, memory problems, and ecosystem incompatibilities. Understand how the team addressed these challenges to keep the project on track, despite the less-than-optimal characteristics for massively parallel processing.