Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [fine-tuning]

For questions related to the concept of fine-tuning a model (e.g. neural network), which is very related to and sometimes used as a synonym for transfer learning.

80 questions
13
votes
1 answer

What is the difference between one-shot learning, transfer learning and fine tuning?

Lately, there are lots of posts on one-shot learning. I tried to figure out what it is by reading some articles. To me, it looks like similar to transfer learning, in which we can use pre-trained model weights to create our own model. Fine-tuning…
Hiren Namera
  • 785
  • 6
  • 20
10
votes
2 answers

Are GPT-3.5 series models based on GPT-3?

In the official blog post about ChatGPT from OpenAI, there is this paragraph explaining how ChatGPT model was trained: We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with…
iMad
  • 203
  • 2
  • 5
4
votes
1 answer

Why fine tuning does not work as well as RAG?

I cannot find a definite answer to this question. Suppose I want to build a QA (question answering) system on a set of personal documents. It looks that RAG (retrieval augmented generation) is the way to go for this task, but I do not understand why…
Thomas
  • 255
  • 2
  • 7
4
votes
2 answers

How can I teach a book to an LLM?

I am trying to find out how I can teach the content of a whole, multiple hundert pages book to an LLM so that it "knows" all details and can be queried, give summaries etc. The book is one consistent story, private and has never been published. I…
dschuld
  • 41
  • 1
  • 3
4
votes
1 answer

What is the difference between fine tuning and variants of few shot learning?

I am trying to understand the concept of fine-tuning and few-shot learning. I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, recently I have seen a plethora of blog posts…
Exploring
  • 371
  • 7
  • 18
4
votes
1 answer

Why aren't the BERT layers frozen during fine-tuning tasks?

During transfer learning in computer vision, I've seen that the layers of the base model are frozen if the images aren't too different from the model on which the base model is trained on. However, on the NLP side, I see that the layers of the BERT…
Bunny Rabbit
  • 141
  • 2
4
votes
1 answer

How to fine tune BERT for question answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from the constitution and no question-context-answer…
Anirban Saha
  • 141
  • 1
  • 3
3
votes
1 answer

When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework. I came across transfer learning, where we fine-tune a pre-trained neural…
Santhosh
  • 143
  • 6
3
votes
1 answer

How do you fine-tune a LLM in theory?

What does it mean to fine-tune a LLM? What can be accomplished with fine-tuning? I am working on cleaning up messy text (call it tweets) into short clean summaries. How can I take advantage of fine-tuning an LLM like LLaMa 3.1 or any of the other…
Lance Pollard
  • 87
  • 7
3
votes
2 answers

What is the difference betwen fine runing and rlhf for llm?

I am confused about the difference betwen fine runing and rlhf for llm. When to use what? I know RLHF need to creating a reward model which at furst rates responses to align the responses to the human preferences and afterward using this reward…
Exploring
  • 371
  • 7
  • 18
3
votes
1 answer

Should I be layer freezing when fine-tuning an LLM?

I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's article: PEFT approaches only fine-tune a small number of (extra) model parameters while freezing most…
multiheadedattention
  • 31
  • 1
  • 3
3
votes
1 answer

Creating a support chat bot for my business

I am trying to create a kind of support bot to answer questions from my clients about specific technical details about WordPress plugins that I sell. The goal is that the /completions API would be fed a prompt, which could be something general like…
digitalzoomstudio
  • 131
  • 3
3
votes
0 answers

What is the difference between prompt tuning and prefix tuning?

I read prompt tuning and prefix tuning are two effective mechanisms to leverage frozen language models to perform downstream tasks. What is the difference between the two and how they work really? Prompt Tuning:…
Exploring
  • 371
  • 7
  • 18
3
votes
0 answers

Why shouldn't batch normalisation layers be learnable during fine-tuning?

I have been reading this TensorFlow tutorial on transfer learning, where they unfroze the whole model and then they say: When you unfreeze a model that contains BatchNormalization layers in order to do fine-tuning, you should keep the…
dato nefaridze
  • 882
  • 10
  • 22
3
votes
0 answers

How does one continue the pre-training in BERT?

I need some help with continuing pre-training on Bert. I have a very specific vocabulary and lots of specific abbreviations at hand. I want to do an STS task. Let me specify my task: I have domain-specific sentences and want to pair them in terms of…
Adrian_G
  • 31
  • 1
1
2 3 4 5 6