Why the company such as openai has not build probabilistic LLM which tells the user "How sure the model is about its answer"?

Question

I am curious why the LLM model is not built to give a probabilistic answer. Whether people are working on it or is it not necessary to be accomplished.

Answer 1

Because it's an ill-defined question.

You might be 100% sure of the wrong answer :)

Answer 2

For language models, the probabilities in question are (to first approximation) of the form "Probability that word i is 'elephant' given the surrounding context", or more generally, the probability that if we randomly select a sentence/phrase/whatever of this length from the set of all possible sentences that we'll get this one. In this distribution, the sentences "The answer is 'elephant'", "An elephant", and "one of those whatsises, like mammoths but less furry" are all separate statements with separate probabilities. The model does not possess a single number that represents the probability that the idea is correct.

Different types of model are more or less amenable to giving certainties of that form, and you could probably define some sort of model that provides a certainty measure over the set of (answer's semantic vector, question's semantic vector) pairs, but that's a decidedly nontrivial project in of itself, and doesn't naturally fall out of the existing math.