There's a video essay that covers this topic: Where are all the Skyrim Killers
The big takeaway is that it's really expensive to do, and it's only gotten more expensive since Morrowind with the expectation of fully voice acted dialogue and complex schedules. Not even Skyrim has full schedules for all NPC's tbh.
Ok, tangent time: I have to wonder if another factor is public perception. Look at discussions on Starfield that pop up and you inevitably get people with... interesting takes.
I've seem people argue that Bethesda's formula is outdated, which is ridiculous. There's plenty of people who still enjoy going back to old games or renewing old formulas. Just look at boomer shooters, which also rely on an "antiquated formula".
Then there's people that haven't picked up a Bethesda game since Skyrim, and complain about how the Creation Engine is outdated. They don't realize that it's gotten a lot of improvements over the years, such as updating it to a 64-bit instruction set for Fallout 4 and Skyrim Special Edition, which has drastically increased stability. Or creating ESL and ESP-FE plugins that effectively add 4095 plugin slots since the original 255 was too restrictive. Or the improvements in gunplay with Fallout 4 and Starfield. Or all the modifications they did to make Starfield even possible.
That being said, I think Bethesda should look into releasing the Creation Engine into open source. There's some impressive work the modding community has done to add in features they care about using SKSE plugins or Community Shaders
