The first three books were incredible, and I think a lot of the show gets a knock-on effect from GRRM's original fan-base. In the same way that a Batman movie gets a good 2-3+ bump from comic book nerds, GoT is buoyed by folks that are enthusiastic about the original material on screen.
Like you said, the acting is stellar, although I don't feel like the younger main cast really hits their stride until S2/S3. I think the adaption, particularly for the first series, is a bit weak and rushed. You lose a lot of the inner monologues and flashbacks that make the perspective-based storytelling so compelling. Instead, you get the periodic (s)expository, where some supporting actor explains the plot to the mains. The visuals definitely improve over seasons (until the big drop-off in S8), although it still never gets to AAA motion picture quality.
I think the best seasons in the series are actually where they depart from the books - S4, 5, 6 - and aren't bound to a writing formula that stumbles in conversion to TV. The main cast is very into the material. The set designs are truly top notch. The wardrobe is particularly stunning. You really have to just stop and soak in those incredible fits.
But I also think GoT is something of a victim of its own success. One of the big appeals of the story and setting is in how it delivers on a dark and gritty medieval setting with periodic twists that catch you off guard in the moment but seem natural enough in hindsight. The amount of intrigue and politics GoT delivers on - particularly in the period when it was being released - resonated with the historical moment. People really were ready to believe Change Was Coming (one way or another). The overarching conflict between morality and practicality in an era when these were genuinely seen as political conversations worth having rang true in the moment. And a lot of subsequent franchises picked up on this, chopping and screwing the underlying story arcs into your Better Call Sauls and Stars War and Marvel shows, until they no longer felt fresh and original.
As to the finale? You shut your damn face about the Sporanos. That ending was spot on.
But whatever you thought of it, this was significantly worse. Everything from the screenwriting to the cinematography takes a huge plunge. There are scenes where you can't see what's on the damned screen because they didn't light it properly. There are scenes where characters mysteriously teleport from location to location without any sensible explanation. There are scenes where you can practically see the actors delivering the lines groan with their clumbsiness.
There's a notable step down from S6 to S7, in so far as the pacing picks up and they start craming more "What a twist!" moments into scenes than is comfortably paced. But they're wrapping things up and that's somewhat forgiveable. But S8 falls straight off a cliff. It genuinely ruins the entire franchise with how bad it lands.