Lessons From Evaluating and Debugging Healthcare AI in Deployment
Explore the challenges and opportunities in translating trustworthy AI from research into healthcare deployment in this Stanford seminar. Delve into insights gained from conducting real-time AI trials and analyzing FDA-approved medical AI systems. Learn about data curation techniques, quantifying data contributions to model success or biases, and continuous real-time testing and explanation of model mistakes. Discover strategies for designing AI to optimize clinician performance and human-AI interactions. Examine topics such as improving clinical trial efficiency, addressing performance issues in dermatology AI, identifying ethnic stereotypes in language models, and using Data Shapley Value for fairness improvement and dataset bias detection. Investigate neuron-level analysis for understanding model behavior, conceptual explanation of mistakes, and natural language model editing to reduce bias. Gain valuable takeaways on the shifting challenges from model training to evaluation and monitoring in healthcare AI deployment. Read more