Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [curse-of-dimensionality]

For questions related to the concept of "curse of dimensionality", which refers to the problem of an exponential increase in volume which occurs when adding extra dimensions to the Euclidean (or input) space. In machine learning and statistics, the curse of dimensionality implies that more data is required to achieve statistical significance, as the number of dimensions of the input increases. The expression was introduced by Richard Bellman in 1957.

For more info, see e.g. https://en.wikipedia.org/wiki/Curse_of_dimensionality.

8 questions
13
votes
4 answers

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder and a decoder (which can be separate neural…
nbro
  • 42,615
  • 12
  • 119
  • 217
2
votes
0 answers

Nearest neighbour search in high dimension retrieves certain points too often

We represent some catalogue items (documents, music tracks, videos, whatever) as vectors embedded in R^d and use them to retrieve nearest neighbours to users query. The typical scenario is that users can input any query and the search results are…
Peter Franek
  • 384
  • 1
  • 4
  • 14
2
votes
1 answer

Why the number of training points to densely cover the space grows exponentially with the dimension?

In this lecture (minute 42), the professor says that the number of training examples we need to densely cover the space of training vectors grows exponentially with the dimension of the space. So we need $4^2=16$ training data points if we're…
Daviiid
  • 585
  • 5
  • 17
1
vote
1 answer

curse of dimensionality v.s. volume of n-dimensional ball v.s. n-dimensional vector embedding space

We know that the performance of machine learning model become worse if we feed the model with a few features and many features (high dimensional data). This is known as the curse of dimensionality. The relationship between performance and the num of…
Muhammad Ikhwan Perwira
  • 800
  • 3
  • 10
1
vote
0 answers

Dimensionality Limitations

I just started learning about AI and have been reading a book called "Foundations of Machine Learning" by Mehryar Mohri so that I can try to create my own. I had a question come up recently: Can I create a machine learning algorithm that can…
Sophia
  • 11
  • 3
1
vote
0 answers

Is it feasible to train a DQN with thousands of input ports?

I designed a DQN architecture for some problem. The problem has a parameter $m$ as the number of clients. In my situation, $m$ is large, $m\in\{100,200,\ldots,1000\}$. For this situation, the number of input ports of the DQN is some few thousand,…
zdm
  • 309
  • 2
  • 9
0
votes
1 answer

High dimensionality - overfitting relationship

I understand both why high dimensionality and overfitting are undesired but recently I came up multiple sources mentioning that High-dimensional data often leads to overfitting ([example][1]) But as far as I understood when more features are being…
Umberto Fontanazza
  • 101
  • 2
0
votes
0 answers

How to tell a model that there are another useful features?

Suppose there is problem of image detection in a CCTV. Look these tensor dimensions below: Input: Image -> (Height, Width, RGB) Output: Mask of Image -> (Height, Width, Num of Object Class) object class for example person, vehicle, and others…
Muhammad Ikhwan Perwira
  • 800
  • 3
  • 10