I think a lot of people base their assumptions on the way cows, goats, sheep and deer have their mammaries positioned between the waist and the rear legs.

(Waist? Seems wrong but midline is also confusing.)

Those also have hooves, and elephants don't, but it's the animals we're used to seeing nurse.

Maybe you're right and it's just a commonly accepted misconception