Source (Bluesky)
Transcript
Here’s an example that Google’s Josh Woodward, VP of the Gemini app, Google Labs, and AI Studio, shared in a blog post about how Personal Intelligence can work. Google also put together a similar example in a video that I’ve embedded below:
For example, we needed new tires for our 2019 Honda minivan two weeks ago. Standing in line at the shop, I realized I didn’t know the tire size. I asked Gemini. These days any chatbot can find these tire specs, but Gemini went further. It suggested different options: one for daily driving and another for all-weather conditions, referencing our family road trips to Oklahoma found in Google Photos. It then neatly pulled ratings and prices for each. As I got to the counter, I needed our license plate. Instead of searching for it or losing my spot in line to walk back to the parking lot, I asked Gemini. It pulled the seven-digit number from a picture in Photos and also helped me identify the van’s specific trim by searching Gmail. Just like that, we were set.
I'm a professional that works with software dev and I can say that LLMs and all the agent stuff keeps flipping between "surprised at what it can do" and "oh another big sign that it's just a really fancy text predictor".
At best, it has reduced the number of times I reach out to colleagues for a problem I solve myself in the process of explaining it and has helped me find obscure settings to fix obscure issues. For coding, I'm still not sure whether or not it saves time. It can write things quickly but it embeds all kinds of assumptions in there and might not even follow instructions.
Like it's safer to think of it as a conversation partner who can pretend to be many different people, including experts in the topics you discuss, but also has ADHD so severe it can switch what it is pretending to be mid-sentence. Even when you ask it to explain what happened after the fact, it just makes up more bullshit because it doesn't have thoughts or awareness, it just predicts tokens.
I ask a question, they give me the wrong answer. I tell them they gave me the wrong answer. They apologize, and then repeat the same mistake in the next answer. Or they give me a different wrong answer. I eventually give up and solve it with a web search.
I don't know if my questions are about really obscure stuff or what, but it's really annoying. Like, I know that they're only predicting tokens, but how hard is it to program them to go "Okay, we've already established that this pattern of tokens is wrong, so I'm not going to include it in the next answer".