Technology

Ah but the thing is, the sources it finds are themselves wrong. They're talking about different things. If it finds the right sources, the information is most likely pretty good.

That's what I was trying to get at, yeah: The word-correlator can't tell the difference because it can't tell which part of the prompt is critical for the difference. It also can't judge whether the sources themselves are accurate about the things they're actually talking about. It find sources that seem related, then aggregates some likely response from those sources and spits it out. By its own metrics and limitations, that has to be "good enough" (but as you say, may be wildly wrong).

My suspicion is that adding more specific terms to the prompt may increase the textual "distance" between the fitting results and the irrelevant ones, thus tilting the source selection towards the more accurate match, but that requires you to know those specific terms in the first place.

It's entirely possible that this could still be tuned somehow

With a general model? I doubt it.

A subject-specific model could probably be tweaked to recognise particular tokens and assign them more weight. Depending on the subject, you might be able to add pre-processing logic to crossreference certain keywords (like model and engine type) with some objective database (like a table describing which pumps a given motor might have, if that makes sense?).

You could also pick the training material more diligently to ensure you only have qualified, reliable sources for that subject. That would require a lot of work, since you also need enough material for the model to be representative.

And you'd have to do that for every specialised field out there.

I wouldn't trust anything that comes out of it unless I know enough to verify the output myself, meaning it's pretty useless to laymen (which is whom a "veracity" bot would be for)

Hit the nail on the head there. For all we know, it'll "correct" facts by doubling down on common misconceptions.