I can't disagree with anything that you're saying here. All I can say is that there is always a balance to be struck between road users - pedestrians, cyclists, cars, everyone (even animals).

So how does it keep happening?

It happens because road users aren't properly segregated. If you look at large construction sites, they have barriers everywhere and it's generally extremely safe. They still have fatal accidents between vehicles/plant and people. The only way to be completely safe is to be separate - but that isn't reasonably practicable everywhere.

My issue here isn't with 20 mph roads. My issue is with a blanket change of all 30 mph roads to 20 mph, without making those roads suitable for 20 mph. Maybe some of these roads already should be 20 mph, but many aren't and shouldn't be, and the ones that aren't will encourage further noncompliance everywhere else.