This is the key - it does not create, it can only copy.

I have asked ChatGPT to write poetry on subjects that I know with great certainty have never had poems written about them.

You can of course shuffle around the meanings of "create" and "copy" to try to accommodate that, but eventually you end up with a "copying" process that's so flexible and malleable that it might as well be creativity. It's not like what comes out of human brains isn't based on stuff that went into them earlier either.