If you're looking for the mathematical side, I'd recommend The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition that said it's quite old school, you won't find any information on why the new techniques work (especially deep learning). Still, if you want to understand bootstrapping, bias variance decomposition, the curse of dimensionality, I'd say this is one of the best books.

I'll also share recommended readings of different EPFL courses that I did for my masters degree :

• Linear algebra and learning from data
• The elements of statistical learning : data mining, inference, and prediction / Friedman
• Understanding Machine Learning / Shalev-Shwartz
• Neural Networks and Deep Learning / Nielsen
• Machine Learning: A Probabilistic Perspective / Murphy
• Pattern Recognition and Machine Learning / Bishop
• Reinforcement Learning / Sutton
• Deep Learning / Goodfellow