Also those thing are highly parallelizable and mainly deal with vector and matrix data, so the same "lots of really simple but fast processing units optimized for vectors and matrix operations working in parallel" that works fine for modern 3D Graphics (for example, each point on a frame image to display on the screen can be calculated in parallel with all the other points - in what's called a fragment shader - and most 3D data is made of 3D vectors whilst the transforms are 3x3 Matrices) turns out to also work fine for things like neural networks were the neurons in each layer are quite simple and can all be processed in parallel (if the architecture of that wasn't layered, GPUs would be far less effective for it).

To a large extent Nvidia got lucky that the stuff that became fashionable now works by doing lots of simple and highly paralellizeable computations, since otherwise it would've been the makers of CPUs that gained from the rise of said computing power demanding tech.