Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [dimensionality-reduction]

For questions related to AI methods of dimensionality reduction (e.g. PCA or autoencoders).

28 questions
13
votes
4 answers

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder and a decoder (which can be separate neural…
nbro
  • 42,615
  • 12
  • 119
  • 217
5
votes
2 answers

What are examples of approaches to dimensionality reduction of feature vectors?

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements. I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine…
doplano
  • 309
  • 3
  • 10
3
votes
1 answer

When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?

I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following. Using PCA to speed up learning Using PCA to reduce the number of features, thus lowering the chances for overfitting Looking at these two…
AfiJaabb
  • 131
  • 3
3
votes
0 answers

What is meant by subspace clustering in MFA?

The basic idea of MFA is to perform subspace clustering by assuming the covariance structure for each component of the form, $\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$, where $\Lambda_i \in \mathbb{R}^{D\times d}$, is the factor loadings matrix…
stoic-santiago
  • 1,201
  • 9
  • 22
3
votes
1 answer

How do AI researchers imagine higher dimensions?

We can visualize single, two, and three dimensions using websites or imagination. In the context of AI and, in particular, machine learning, AI researchers often have to deal with multi-dimensional random vectors. Suppose if we consider a dataset of…
hanugm
  • 4,102
  • 3
  • 29
  • 63
2
votes
0 answers

How to reduce the dimensionality of the actions in RL

I have a single-agent RL model in which the dimension of the dimension of the action space is $70$. This action space is too big and the deep RL agent is not learning properly. The boundaries of the action space are $-1$ and $1$. My question is, how…
Leibniz
  • 69
  • 5
2
votes
1 answer

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours search. Now the next task, where I am facing a problem…
VEDANT JOSHI
  • 31
  • 1
2
votes
1 answer

Why does PCA of the vertices of a hexagon result in principal components of equal length?

I do PCA on the data points placed in the corners of a hexagon, and get the following principal components: The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be greater in the horizontal direction than in the…
Vladislav Gladkikh
  • 534
  • 6
  • 14
2
votes
0 answers

Estimating dimensions to reduce input image size to in CNNs

Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the final pooling layer within the CNN network such…
Prishita Ray
  • 51
  • 4
2
votes
0 answers

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in total) input-parameter-output match. My goal is to…
t-smart
  • 121
  • 2
2
votes
2 answers

How can I use autoencoders to analyze patterns and classify them?

I generated a bunch of simulation data from a complex physical simulation that spits out patterns. I am trying to apply unsupervised learning to analyze the patterns and ideally classify them into whatever categories the learning technique…
Pavan Inguva
  • 71
  • 1
  • 2
2
votes
0 answers

Clustering of very high dimensional data and large number of examples without losing info in dimensions

I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from $50$ genres. Each song has characteristics - some of which are common across all or most genres,…
Shirish
  • 403
  • 3
  • 11
1
vote
0 answers

How would PCA change if we center data using the coordinate-wise median instead of the mean?

In standard Principal Component Analysis (PCA), we center the data by subtracting the mean of each coordinate before computing the covariance matrix and solving for the principal components. I am wondering: what would happen if, instead of centering…
the2second
  • 11
  • 1
1
vote
1 answer

Is it possible to use attention layer as a sort of filter of input data before passing it further to the network?

Is it possible to use attention layer as a sort of filter of input data before passing it further to the network? Is it possible to use it to reduce the dimension of the input (similar as PCA, for example - only attention layer would be trained with…
janek nowaczek
  • 11
  • 1
1
vote
1 answer

What is $\mathbf{S}$ (sample covariance matrix) in image compression based on PCA?

If the feature vector is $\mathbf{x} \in \mathbb{R}^{d}$, then to apply PCA we first need to construct the "sample covariance…
DSPinfinity
  • 1,223
  • 4
  • 10
1
2