I'm not atheist, nor Muslim, nor rightist, but you know, the difference is easy to make. Do you criticize Islam because you think it's a bad way of life? Or do you hate the people who follow this way? One can ciriticize a religion and/or the religion. But you become an ally of far-right when you begin to treat people you disagree with differently.
Me, for example, I don't like the islamic scarf. One of my best friends wears one. We can disagree and still respect and like each other.