If that were legal, I'd absolutely be worried, you make a good point.

Even Doctor need special additional qualifications to do things like diagnose illnesses via radiographic imagery, etc. Specialised AI is making good progress in aiding these sorts of things, but a generalised and very poor AI like ChatGPT will never be legally certified to do this sort of thing.

Once we have a much more effective generalised AI, things will get more interesting. It'll have to prove itself thoroughly though, before being certified, so it'll still be a few years after it appears before we see it used in clinical applications.