Our journey to apply deep learning for giant hogweed eradication
5
About giant hogweed
6
Project Overview
7
Data volumes
8
Various specialty
9
Challenges of the project
10
Architecture overview
11
Data flow-1/4: Data preparation phase
12
Assistance tool for data preparation
13
Inference processing by Apache Spark
14
Data flow-4/4: Data analysis phase
15
Lessons Learned from a consideration of architecture design
16
Common view in Machine learning system
17
Scaled-ML systems
18
ML Application with scalability on our architecture
19
Model development vs. Model operating
20
Dev-friendly to Dev-friendly
21
Dev-friendly to Ops-friendly
22
Use case: Uber implemented ML Ops on Spark
23
Ops-friendly to Ops-friendly Patterns that selected a toolset familiar to Model Operator for both model development and operation
24
Use case: Twitter leveraged Scala for feature engineering
25
Example software: BigDL by Intel
26
Patterns of software choices Reprint Four patterns of the workflow regarding combination of Model Development phase and Model Operating phase
27
Architecture and data pipeline ver.2.0
28
Tips: Abstraction of functions used in applications
29
Tips: Detecting and storing deterioration of confidence
Description:
Explore a conference talk detailing the application of deep learning and big data technologies in eradicating giant hogweed. Dive into the project overview, challenges faced, and architectural design of a system combining various specialties and large data volumes. Learn about data preparation, inference processing with Apache Spark, and data analysis phases. Gain insights into scaled machine learning systems, the balance between model development and operation, and strategies for creating dev-friendly and ops-friendly environments. Discover real-world use cases from Uber and Twitter, and examine patterns for selecting toolsets and software choices. Acquire valuable tips on function abstraction and detecting confidence deterioration in applications.
Lessons Learned from Big Data and AI/ML Collaboration for Giant Hogweed Eradication