My personal theory:

First off, raw milk does taste noticably different than pasteurized and homogenized milk you find at the store.

Pasteurization: heating the milk triggers the unfolding of proteins (Denaturation). This is what kills the bacteria but can also change the flavor of the milk.

Homogenization. This process breaks up the fat into smaller segments so they stay in solution in the milk. The result is a less creamy flavor.

People instinctually associate flavor with nutritional value. They think that better flavored food = better for you. This sort-of works in tomatoes and a few other fruit/vegetables. However taste perception is a complex blend of genetics, environmental conditions, and psychology. So the results are inherently unpredictable and completely unreliable.

The unpasteurized crowd all fall for the 'it tastes better so it must be better". They then make all sorts of excuses to justify their instinct. " Big corporate milk is evil!!" Blah blah blah.