Linux

Using GTA online as an example of anything secure isn't great, given it was losing to cheat engine early on. Surely if they put any effort into anticheat at all, cheat engine would fail

No, Heuristic Analysis is deciding what data is likely, what data is unlikely, and what data is impossible, and then deciding, on that scale, the where the data the player is generating resides.

In short: Humans have natural variations in everything they do, even the top 0.0001% of players. So let's say you want to tackle aimbots in an FPS.

The first thought would be track the number of headshots, and then if a player gets 100% headshots they're labeled a cheater -- but that isn't accurate because of players like the streamer Shroud. So let's be smarter. Let's analyze the median player based on data from every player -- not their headshots, not where they shoot, but how they move the cursor to the opponent to shoot.

An aimbot will do a simple mathematical formula to decide how to aim at the target; i.e. if we imagine a 2d grid (centered at 0,0; squared limits of 100) on the screen and the player's crosshair is at 0,0 and there's an enemy at 50,50; then a bot would do something like (complete pseudocode:)

While CrosshairPosition(y) does not equal EnemyPosition(y):
    Move mouse up (i.e. +y) by 1
    While CrosshairPosition(x) does not equal EnemyPosition(x):
        Move mouse right (i.e. +x) by 1
Fire()

This results in a predictable and perfectly diagonal move towards the enemy. Now actual humans cannot do this. It doesn't matter how fine of motor skills they have, period. It is impossible for a human to even accidentally move like this. So we place this in the 'impossible' end of the spectrum.

If a player does too many unlikely or impossible actions, flag them for review, and ban them that way. Or, just ban the ones doing objectively mathematically impossible things.

Heuristic Data Analysis requires actual humans actually thinking about what is and isn't possible in a game, understanding how cheats AND the game actually work, and then defining the spectrum, and then implementing and constantly tweaking it to minimize false positives while maximizing those that tweak their bots to get around the analysis.

Because of this it's expensive, relatively speaking, than paying a (statistically Israeli) anti cheat company to install spyware on their behalf.