Explore the challenges of class imbalance and conflicting metrics in machine learning for side-channel evaluation in this 21-minute conference talk presented at the Cryptographic Hardware and Embedded Systems Conference 2019. Delve into the importance of Hamming Weight (HW) and the impact of imbalanced data on machine learning models. Learn about various data sampling techniques, including random under sampling and random oversampling with replacement. Examine experimental results from two datasets and understand the implications of different evaluation metrics, particularly the relationship between Success Rate/Guessing Entropy (SR/GE) and accuracy. Gain valuable insights and takeaways for improving machine learning approaches in side-channel analysis.