Am I acting paranoid for suspecting that somebody deliberately tampers with these statistics?

I read somewhere that everybody except for white men voted in greater proportions for Trump four years ago than they did eight years ago, and I found that very hard to believe; I never saw anybody explain that, unless maybe the general turnout was smaller.