For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning.
Questions tagged [embeddings]
90 questions
40
votes
5 answers
What is the difference between latent and embedding spaces?
In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" and "embedding space" occur in several contexts. More specifically, an embedding can refer to a…
nbro
- 42,615
- 12
- 119
- 217
18
votes
3 answers
Why are embeddings added, not concatenated?
Let's consider the following example from BERT
I cannot understand why "the input embeddings are the sum of the token embeddings, the segmentation embeddings, and the position embeddings". The thing is, these embeddings carry different types of…
nalzok
- 401
- 6
- 10
14
votes
3 answers
How to determine the embedding size?
When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden vectors (or embeddings), but I wonder if there…
Lerner Zhang
- 1,065
- 1
- 9
- 22
5
votes
2 answers
What is the intuition behind how word embeddings bring information to a neural network?
How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer?
I understand how the word embedding carries some semantic meaning, but it seems that this information would…
silkAdmin
- 209
- 1
- 3
5
votes
2 answers
Is an embedding a representation of a word or its meaning?
What does the term "embedding" actually mean?
An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. Which one is actually correct?
Or is there anything like: A…
hanugm
- 4,102
- 3
- 29
- 63
4
votes
2 answers
How can I teach a book to an LLM?
I am trying to find out how I can teach the content of a whole, multiple hundert pages book to an LLM so that it "knows" all details and can be queried, give summaries etc. The book is one consistent story, private and has never been published. I…
dschuld
- 41
- 1
- 3
4
votes
3 answers
How embeddings learned from one model can be used in another?
In the website the following explanation is provided about Embedding layer:
The Embedding layer is initialized with random weights and will learn
an embedding for all of the words in the training dataset.
It is a flexible layer that can be used in…
Oculu
- 53
- 1
- 3
4
votes
2 answers
What is the difference between representation and embedding?
As I searched about this two terms, I found they are somehow like each other, both try to create a vector from raw data as I understood. But, what is the difference of this two term?
aliiiiiiiiiiiiiiiiiiiii
- 143
- 1
- 3
4
votes
1 answer
What are knowledge graph embeddings?
What are knowledge graph embeddings? How are they useful? Are there any extensive reviews on the subject to know all the details? Note that I am asking this question just to give a quick overview of the topic and why it might be interesting or…
nbro
- 42,615
- 12
- 119
- 217
4
votes
1 answer
How does the embeddings work in vision transformer from paper?
I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to Flatten the 3-D (16,16,3) patch to pass it into a Linear layer to get what they call "Liner Projection". After passing from the…
Deshwal
- 263
- 3
- 11
3
votes
1 answer
Match two paragraphs of text
I'm building a friend finder app and I need to match people based on a paragraph of text. Here is an example of what I mean:
Person A:
I love walking and going to the beach, I also love reading and keeping active. I'm very allergic to dogs, so I…
Dom
- 31
- 3
3
votes
0 answers
How to work with multiple embeddings?
This is a conceptual gap that I have concerning embeddings, and would really appreciate some help closing it.
I understand when you embed a corpus for, let's say, a question-and-answer task you can then encode your query to find documents within…
Ian Murray
- 31
- 1
3
votes
1 answer
Creating a support chat bot for my business
I am trying to create a kind of support bot to answer questions from my clients about specific technical details about WordPress plugins that I sell.
The goal is that the /completions API would be fed a prompt, which could be something general like…
digitalzoomstudio
- 131
- 3
3
votes
1 answer
What is an "input embedding" in the context of NLP?
When reading about NLP, I saw it said that "input embeddings" are a main element of encoder-decoder learning frameworks for sequence modelling. What is an "input embedding" in the context of NLP?
The Pointer
- 611
- 5
- 22
3
votes
1 answer
In the machine learning literature, what does it mean to say that something is "embedded" in some space?
In the machine learning literature, I often see it said that something is "embedded" in some space. For instance, that something is "embedded" in feature space, or that our data are "embedded" in dot product space, etc. However, I've never actually…
The Pointer
- 611
- 5
- 22