This is a good point for not choosing too small. I've made a couple of accounts, and it looks like when a servers crosses that 1,000 or 2,000 user mark you start getting much better consistency than the micro instances with only a few hundred users.
I usually find that I have to reload a few times if I'm the first person to try to subscribe to a community. That happens uncomfortably too often if the instance is small. Even then, it can take a days or possibly never to properly federate.
I'm sure these issues will be fixed, but for now, I'd like myself a small instance but not too small so as to avoid issues with consistency.
All refers to everything that your instance knows about. Your instance only retrieves data for which users are actually subscribed.
All can be weird on small instances if the user subscriptions don't have a nice distribution.