1

As far as I understand ChatGPT has been trained on a vast array of data, and it does understand questions; but it seems to never ask. Even if a person would ask clarifying questions (that I assume are in the train set) ChatGPT doesn't, opting instead to invent context or just say "X depends on Y, Z"...

Not asking questions seems trained into the network, but I am not sure how one would go about training a model not to generate questions in such a way that not even in DAN mode it doesn't. I understand that for toxic language GPT-3 uses human raters to generate a train set and then optimizes for non toxic behavior, but it seems to specific to be used for the general concept of questions.

EmmanuelMess
  • 227
  • 3
  • 16

1 Answers1

1

Large Language Models are fundamentally trained to predict the next word in a sequence and continue this process iteratively. They're great at completing an unfinished conversation, akin to seamlessly continuing a novel mid-chapter. However, AI assistants like ChatGPT take this technology a step further by structuring/prompting it to respond directly and concisely as if they were individuals.

I mention this because while the LLMs may appear to have "agency," it is more accurate to frame it as a being that tries to please you by saying most likely (or the most rewarding) sentences. Their goal isn't to understand you deeply; rather, they're essentially mimicking an overly confident participant who tries to keep the conversation flowing even without full comprehension. Their reward mechanism emphasizes producing plausible or appealing responses rather than genuine understanding.

I think the training data plays a key role in this too. Clarifying questions are abundant in real-time interpersonal interactions but relatively rare in polished or formal written content, which constitutes a significant portion of an LLM’s training data. We might overestimate how often clarifying questions appear in the texts LLMs learn from.

SHJ
  • 11
  • 1