There would need to be a stupid complex network of regulations and inspectors to do it right.

This is how all regulations work (for the most part). Also note that there's no such thing as "too much" or "too little" regulation. There's just bad regulations and good regulations.

Examples of good regulation:

• Car emissions standards
• Appliance energy efficiency requirements
• Safety regulations (all kinds)
• Loads of regulations related to banking

Examples of bad regulation:

• Requiring certification for things that don't need it (e.g. braiding hair)
• Requiring drug tests in order to receive government benefits
• Banning necessary medical procedures and medications based on religious beliefs
• Loads of regulations related to banking

Note: I work for a huge bank and our executives bitch about the cost of compliance all the time. Make them bitch more. Of all the things that need regulation, banking is of the greatest need. Never trust any financial system or transaction that isn't heavily regulated! There's an infinite number of ways to get screwed via banking and if it weren't for regulations they would screw you and everyone else as much as they possibly could. History is full of examples.