If it wasn't historically relevant, why would they use the word on their page? Coed is relevant because women used to segregated or not be eligible for higher education at all. He's taking a shot at women, not people of colour. He yearns for the times women "knew their place" (ugh, I feel gross just typing that) and didn't seek education or a career.

In any case, that clip sounds to me like "coloured" only a tiny bit more than it sounds like "college", which is to say, not much.