An AI doesn't understand. It has an internal model which produces outputs, based on the training data it received and a prompt. That's a different cathegory than "understanding".

Is it? That's precisely how I'd describe human understanding. How is our internal model, trained on our experiences, which generates responses to input, fundamentally different from an LLM transformer model? At best we're multi-modal, with overlapping models which we move information between to consider multiple perspectives.