In this introductory talk, I aim to give a broad overview over machine learning (ML) and related techniques as they have been applied to problems in materials science. The fundamental tradeoff between uncertainty and bias inherent to learning shall serve as our guide. It explains why large-scale general purpose models, while valuable, have had limited impact in physics, and why on the other hand simple ML models tailored to specific problems have led to breakthroughs to molecular dynamics and quantum field theories. Time permitting, we will turn to physical interpretations and phase transitions in the tradeoff itself.