I don't think speed is the main factor, I think it's entitlement. With public transportation, you might have to go somewhere specific to get your ride, rather than it being just outside. Then you wait til a certain time, share the ride with others, the path and speed arent in your control so it may not be the most efficient route. With a car, you can go when you want, where you want and how you want. Until you know, others get in the way. If something is making you drive slightly slower, you now have less control.

I also think driving in general is an unpleasant experience. For most of the time driving, you are staring at asphalt, concrete and other cars, nothing is fun to look at. You have to focus constantly and any distractions become annoyances. Because of cars everything is so far away it always requires the car to do ANYTHING. For all the supposed freedom, you are constantly being told what to do. This is the speed limit, this is a stop sign, this is your designated lane, you have to yield. At every point you run into people and the more you run into the less you can do what you want. And those people can act like don't understand rules, aren't paying attention, interpret the rules differently, think they don't apply to them, know they can get away with it because the rules aren't heavily enforced or they are selectively enforced. Sometimes the rules don't always make the most sense, they are the way they are so it's easier to assign blame to someone in a insurance claim or court case. Or sometimes the rules make sense for someone else, but not your entitlement machine.