I have not played it, but people rave about The Outer Wilds and it sounds like it fits your needs.

It's not at all my thing, but search the term "walking simulator" to find stuff like that.

Have you tried some of the games you're talking about on the "story" difficulty modes? Most have moved to calling it something like that instead of "easy", and I'm at the opposite extreme, but a lot of them are designed to let you experience the world and story without the pressures of combat.

If you have access to a switch, Mario Odyssey or Kirby and the Forgotten Land have some "combat" but you can skip a lot of it, and they're made to be beatable by kids. Other 3D platformers exist in similar veins as well.

I don't see Thief mentioned yet. It's old but it was called a "first person sneaker" by some reviewer back in the day.

I'm seeing something on Steam that looks like it but with a 2/2014 release date, which seems newer that I remember. Apparently there's a bunch of DLC too, so you might consider buy a bundle.

Firewatch
Well-made story-focused, no combat whatsoever. Very few puzzles, almost exclusively about exploring.

What Remains of Edith Finch
More "puzzley", but lets you discover things at your own pace.

Right off the top of my head is Abzu. Not a hard game, simple puzzles, but really nicely crafted environments and experience. It's a bit short but it was well worth the experience. It's quite an old game by now so probably not hard to get it on sale.

Seems like you want something like a walking simulator, maybe a point and click game?

There's Journey, I've never personally played it but from what I gather it's mostly walking, jumping and exploring.

Have you also tried Minecraft? You can play on peaceful, so no combat at all, you'd be mostly exploring.

Maybe Subnautica? There's a creative mode, you don't need to worry about survival and dying, you can build indefinitely.

Gone Home is pretty neat. I usually need secondary gameplay loop to placate my short attention span but this one gripped me through presentation and atmosphere alone.

I really enjoyed The Planet Crafter! It's a crafting game bringing a barren world to fully lush environment. Lakes will form, grasses start to grow. It's very peaceful and visually appealing.

Myst and riven will lock your exploration behind puzzles. Outer wilds is a good bet!