Explore deep learning applications in scientific computing through two case studies highlighting the gap between theory and practice. Delve into high-dimensional function approximation and inverse problems for imaging, examining limitations of current approaches in stability, generalization, and practical performance. Discover recent theoretical advancements demonstrating the potential of deep neural networks to match best-in-class schemes in both settings. Gain insights into achieving robust, reliable, and improved practical performance in scientific computing using deep learning techniques. Learn about challenges in parametric modeling, limited performance for smooth function approximation, and unpredictable generalization in inverse problems. Understand the universal instability theorem and its implications for deep learning in scientific applications.