You seem to be conflating two unrelated points. When people say "tourists need to go home" or "we need less tourists," they aren't referring to some philosophical argument for the existence of tourism as a gauge of economic health for the middle class.

These phrases are almost exclusively speaking in regards to tourism's negative effects on the local environment and population. Heavy tourism has a tendency to cannibalize an areas resources and then alienate or push out the native population.