Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [generalization]

For questions related to the concept of generalization in computational learning theory and machine learning.

45 questions
16
votes
1 answer

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our results. Instead, we do 10 fold cross validation…
Philipp Cannons
  • 161
  • 6
15
votes
1 answer

What are the state-of-the-art results on the generalization ability of deep learning methods?

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite seemingly overfitting training data. A lot of people in…
Shirish
  • 403
  • 3
  • 11
14
votes
3 answers

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization process overall?
kenorb
  • 10,525
  • 6
  • 45
  • 95
11
votes
5 answers

Why can neural networks generalize at all?

Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they seem particularly apt at learning the right…
Nico A
  • 211
  • 2
  • 8
7
votes
1 answer

When exactly is a model considered over-parameterized?

When exactly is a model considered over-parameterized? There are some recent researches in Deep Learning about the role of over-parameterization toward generalization, so it would be nice if I can know what exactly can be considered as such. A…
Phúc Lê
  • 191
  • 1
  • 5
5
votes
2 answers

Is pooling a kind of dropout?

If I got well the idea of dropout, it allows improving the sparsity of the information that comes from one layer to another by setting some weights to zero. On the other hand, pooling, let's say max-pooling, takes the maximum value in a…
nsaura
  • 258
  • 2
  • 7
5
votes
1 answer

How can my Q-learning agent trained to solve a specific maze generalize to other mazes?

I implemented Q-learning to solve a specific maze. However, it doesn't solve other mazes. How could my Q-learning agent be able to generalize to other mazes?
lrosique
  • 63
  • 4
5
votes
3 answers

How can the generalization error be estimated?

How would you estimate the generalization error? What are the methods of achieving this?
kenorb
  • 10,525
  • 6
  • 45
  • 95
5
votes
1 answer

What are the techniques for detecting and preventing overfitting?

I'm worrying that my neural network has become too complex. I don't want to end up with half of the neural network doing nothing but just take up space and resources. So, what are the techniques for detecting and preventing overfitting, to avoid…
kenorb
  • 10,525
  • 6
  • 45
  • 95
4
votes
1 answer

Does DQN generalise to unseen states in the case of discrete state-spaces?

In my understanding, DQN is useful because it utilises a neural network as a q-value function approximator, which, after the training, can generalise to unseen states. I understand how that would work when the input is a vector of continuous values,…
Redox
  • 43
  • 2
4
votes
1 answer

Is there a way to ensure that my model is able to recognize an unseen example?

My question is more theoretical than practical. Let's say that I am training my cat classifier with a dataset that I feel is pretty representative of cat images in general. But then a new breed of cat is created that is distinct from other cats and…
mdurrant
  • 41
  • 2
3
votes
2 answers

When exactly am I overfitting -- contradicting metrics

I am training an object detection machine learning pipeline. Among the many metrics provided out of the box by tensorflow object detection API, I look at total_loss and DetectionBoxes_Precision/mAP@.75IOU: Here the x-axis is the number of steps…
user1091141
  • 133
  • 5
3
votes
1 answer

Is there a notion of generalization in unsupervised learning?

I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem. So, I started to wonder if there is an analogous version of "generalization" in Unsupervised Learning?…
Marcel
  • 133
  • 3
2
votes
1 answer

Are we really misunderstanding VC theory as arXiv:2205.15549 suggests?

arXiv:2205.15549 claims that the machine learning community misunderstood VC (Vapnik–Chervonenkis) theory and VC-theoretical understandings are sufficient to understand the double descent phenomenon. The matter seems fairly weird, since I see that…
Neijal Kanderbalt
  • 31
  • 2
2
votes
2 answers

Do representations introduce an inductive bias?

I am trying to get a better understanding of inductive biases used in machine learning. I understand inductive biases as the assumptions encoded into the learning algorithm which make it prefer one hypothesis over another. Architectural choices are…
Antonios Sarikas
  • 231
  • 1
  • 7
1
2 3