Housing market collapses don't happen in low confidence events. They happen due to overconfidence leading to excessive lending at high rates and then a sudden sharp decrease in capital availability with which to repay those loans.
At any given time you might have high rates and decrease in capital, or you might have excessive lending, but it's difficult to get all of those factors at once. Especially given how sensitive investors are to such a collapse, as some of them would withdraw investments far ahead of any large events, which would lead to stability in the market as the eventual crash ends up being far less steep.