6

I know that large language models like GPT-3 are trained simply to continue pieces of text that have been scraped from the web. But how was ChatGPT trained, which, while also having a good understanding of language, is not directly a language model, but a chatbot? Do we know anything about that? I presume that a lot of conversations was needed in order to train it. Did they simply scrape those conversations from the web, and where did they find such conversations in that case?

Rexcirus
  • 1,309
  • 9
  • 22
HelloGoodbye
  • 323
  • 1
  • 12

1 Answers1

6

The key ingredient is called Reinforcement Learning from Human Feedback (RLHF), that is having humans rate the model answers and use the feedback to guide the model training.

enter image description here

The official blog explains this fairly well.

Rexcirus
  • 1,309
  • 9
  • 22