Explore the potential benefits and risks of machine learning in cancer diagnosis through this 21-minute conference talk from the AAAS Annual Meeting. Delve into the challenges of using artificial intelligence for medical diagnoses, particularly in cases where the "ground truth" is uncertain. Examine the promises of ML technology in transforming medicine, including faster, more accurate, and cost-effective diagnoses. Investigate the important potential harms associated with ML-driven cancer diagnosis, focusing on the gold standard problem in external validation. Learn about proposed solutions to maximize the benefits of this powerful technology while mitigating risks. Gain insights into the accuracy of pathologists in diagnosing melanoma and understand the complexities of establishing a reliable external standard for AI-assisted diagnoses. Discover the speaker's recommendations for implementing guardrails to ensure responsible use of machine learning in cancer diagnosis.