Yeah, math leads to logic; it's a slippery slope.

If you consider 'Math' as a stand-in for 'Science' and 'empirical evidence' their fear of it becomes a little more cogent.

Of course they fear 'math' as it's the basis for things like 'statistics' and 'empirical evidence' that would clearly demonstrate that all their beliefs are factually incorrect or completely without warrant.