PSA: DM's, never imitate this
Back in the day, PC's would get strongholds and followers when they hit a certain level. Every crap DM (myself included) would think is was a great idea for monsters to sack one while the PC was away. It was never any fun.
When a player puts a little effort work into something, like a castle, house, or cart, don't trash it without their buyin. It's like a thief stealing their items when they were sleeping.
There are better ways to use these new settings in the campaign.