It's important to be able to reason about performance, even in ordinary business computing. If you pick the wrong data structure or algorithm, you can create memory problems, speed bottlenecks, etc. Even if you always use libraries it helps to understand the algorithms and structures they use so you can build efficient software.

And in game programming some knowledge of algorithms and data structures is very valuable. You often have to direct the behaviour of many objects at once in real time and, efficiency is really important.

Without this background knowledge you won't recognize how the problem you are faced with is best addressed by a particular well-known algorithm. Recently, for example, I worked on a business problem that turned out to be tractable by arranging data into a graph and finding optimal pathways through the graph. This wasn't obvious from the initial problem description. And for a game I needed to learn about N-body algorithms and how to efficiently approximate gravitational forces between many objects.

I wouldn't say you need advanced math to do these kinds of programming, but you do need to know when it's time to defer to other people's expertise and tried-and-tested data structures and algorithms.