then I feel like cameras to detect what's in your fridge is pathetically easy in comparison

But you're skipping over a huge amount of context that's missing. It's context we (as humans) take for granted. What's the difference between a jar and a bottle? Is the cream cheese in a tub or in a little cardboard container? Then it would need to be able to see all items in a fridge, know the expiration dates for each thing, know what you want to get, how quickly something gets used, etc.

Some of those things are more straightforward, and some of them need data well beyond "this container has milk". The issue isn't processing all the data, but acquiring it consistently and reliably. We humans are very chaotic with how we do stuff in the physical world. Even the most organized person would throw off an AI system every so often. It's the reason self driving cars are not a reality yet and won't be for a while.