Machine Learning

(I started as a data science years ago, but quickly turned data engineer and have definitely lost touch, but I can help with some generic answers: )

"Brute forcing every algorithm" sounds a bit bad, but it's usually not that bad: in reality when you're "done" you probably want to retrain when new data comes in every so often, so trying out a few algorithms isn't too big of a waste.

That said, with experience comes some knowledge about the tradeoffs of each algorithm/model. For instance, neural nets are universal approximators, so they can in theory model anything, but they need a lot of data to make this happen. In addition, NNs are bad for explainability : when you want to tell e.g. a customer why you're recommending them xyz movie, "neuron 231 activated strongly" is not really useful.

While a linear regression can in theory work with just a handful of samples, but is only useful if your data actually roughly matches a line. This is really good for explainability however: if you see that particles with high energies are more likely to interact, then your linear regression will show that, and it's a clear correlation between those variables.

Apart from amount of training data needed and explainability there's lots of other variables that play a role like how fast the model decays (e.g. anything trained on news data may be outdated in days if not hours, while physics doesn't really change) and as such how often it needs retraining; how easy it is to update vs retrain from scratch each time; cost (in time usually) of training and inference; what you're optimising for (false positives, false negatives, accuracy, precision, etc).

It's not uncommon to just shotgun a bunch of models (and a bunch of variation of hyperparameters of those models) at a dataset and just pick whatever seems to perform best on a validation dataset.