Questions tagged [contrastive-learning]
8 questions
10
votes
1 answer
What is the difference between the triplet loss and the contrastive loss?
What is the difference between the triplet loss and the contrastive loss?
They look same to me. I don't understand the nuances between the two. I have the following queries:
When to use what?
What are the use cases and advantages or disadvantages…
Exploring
- 371
- 7
- 18
2
votes
2 answers
Definition of negatives in NT-Xent loss
I'm trying to understand few details about NT-Xent loss defined in SimCLR paper(link). The loss is defined as
$$\mathcal{l}_{i,j} = -\log\frac{\exp(sim(z_i,z_j)/\tau)}{\sum_{k=1}^{2N}\mathbb{1}_{[k\neq i]} \exp(sim(z_i,z_k)/\tau)}$$
Where $z_i$ and…
James Arten
- 307
- 1
- 12
2
votes
2 answers
Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?
I am looking at this lecture, which states (link to exact time):
What the triplet loss allows us in contrast to the contrastive loss is
that we can learn a ranking. So it's not only about similarity, being
closer together or being further apart,…
Gulzar
- 789
- 1
- 10
- 27
1
vote
1 answer
Embedding Quality of Transfer Learning model vs Contrastive learning model
I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.
The downstream task is a classification problem.
Transfer Learning
Due to limited data, I tried to use…
Raj Rajeshwari Prasad
- 123
- 3
1
vote
1 answer
What is the difference between Mean Teacher and Knowledge Distillation?
I recently read two papers:
BYOL Bootstrap your own latent: A new approach to self-supervised Learning
DINO Emerging Properties in Self-Supervised Vision Transformers.
I am confused about the terms Mean Teacher in BYOL and Knowledge…
Đặng Huy Hoàng
- 11
- 2
1
vote
0 answers
How to use K-means clustering to visualise learnt features of a CNN model?
Recently, I was going through the paper Intriguing Properties of Contrastive Losses. In the paper (section 3.2), the authors try to determine how well the SimCLR framework has allowed the ResNet50 Model to learn good quality/generalised features…
VEDANT JOSHI
- 31
- 1
0
votes
0 answers
how to use contrastive loss function for multi label classification?
I have a multi label classification problem, where I was initially using a binary cross entropy loss and my labels are one hot encoded. I found a paper similar to my application and have used contrastive loss function, but I am not sure how to use…
ThinkPad
- 41
- 6
0
votes
1 answer
Relation of Contrastive Learning and intermediate layers of classifier
Contrastive Learning learns representations of data such that similar samples (positive samples) are close to each other in an Euclidean space, and dissimilar ones (negative samples) are further from each other. If we have labeled samples, then we…
Liubove
- 11
- 3