For questions related to principal component analysis (PCA), which is commonly used in machine learning for dimensionality reduction.
Questions tagged [principal-component-analysis]
26 questions
5
votes
2 answers
What are examples of approaches to dimensionality reduction of feature vectors?
Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.
I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine…
doplano
- 309
- 3
- 10
3
votes
1 answer
Subspace of Autoencoder's Latent Vectors and PCA
I read that a linear decoder spans the same vector space as PCA when the loss function is MSE.
Can someone explain why this is? I can understand that when the decoder is linear it will span the space of the latent vectors. But what makes it the same…
SAS2Python
- 151
- 2
3
votes
1 answer
When using PCA for dimensionality reduction of the feature vectors to speed up learning, how do I know that I'm not letting the model overfit?
I'm following Andrew Ng's course for Machine Learning and I just don't quite understand the following.
Using PCA to speed up learning
Using PCA to reduce the number of features, thus lowering the chances for overfitting
Looking at these two…
AfiJaabb
- 131
- 3
3
votes
0 answers
Looking for the proper algorithm to compress many lowres images of nearby locations
I have an optimization problem that I'm looking for the right algorithm to solve.
What I have: A large set of low-res 360 images that were taken on a regular grid within a certain area. each of these images is quite sparsely sampled and each of…
matthias_buehlmann
- 251
- 1
- 6
3
votes
1 answer
Do the eigenvectors represent the original features?
I've got a test dataset with 4 features and the PCA produces a set of 4 eigenvectors, e.g.,
EigenVectors: [0.7549043055910286, 0.24177972266822534, -0.6095588015369825, -0.01000612689310429]
EigenVectors: [0.0363767549959317, -0.9435613299702559,…
Crizly
- 131
- 1
2
votes
1 answer
What are alternatives to PCA for time series data?
I have some data (20 stock price time series) and want to compare different approaches for dimensionality reduction other than PCA (I want to fit only 2 variables in my AR model). I've tried autoencoders, but their reproduction error is very high.…
IKNv99
- 31
- 3
2
votes
1 answer
Why does PCA of the vertices of a hexagon result in principal components of equal length?
I do PCA on the data points placed in the corners of a hexagon, and get the following principal components:
The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be greater in the horizontal direction than in the…
Vladislav Gladkikh
- 534
- 6
- 14
2
votes
0 answers
Estimating dimensions to reduce input image size to in CNNs
Considering input images to a CNN that have a large dimension (e.g. 256X256), what are some possible methods to estimate the exact dimensions (e.g. 16X16 or 32X32) to which it can be condensed in the final pooling layer within the CNN network such…
Prishita Ray
- 51
- 4
2
votes
2 answers
How does PCA work when we reduce the original space to 2 or higher-dimensional space?
How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case.
$$\begin{array}{ll} \text{maximize} & \mathrm{Tr}\left(…
VN Pikachu
- 21
- 3
1
vote
1 answer
Why does PCA work well while the total variance retained is small?
I'm learning machine learning by looking through other people's kernel on Kaggle, specifically this Mushroom Classification kernel.
The author first applied PCA to the transformed indicator matrix. He only used 2 principal components for…
Bicheng
- 111
- 1
1
vote
1 answer
How to perform PCA in the validation/test set?
I was using PCA on my whole dataset (and, after that, I would split it into training, validation, and test datasets). However, after a little bit of research, I found out that this is the wrong way to do it.
I have few questions:
Are there some…
LVoltz
- 131
- 1
- 6
1
vote
0 answers
How would PCA change if we center data using the coordinate-wise median instead of the mean?
In standard Principal Component Analysis (PCA), we center the data by subtracting the mean of each coordinate before computing the covariance matrix and solving for the principal components.
I am wondering: what would happen if, instead of centering…
the2second
- 11
- 1
1
vote
1 answer
Unsupervised learning of coordinate transformation (e.g. Cartesian to Spherical)
Is it possible to learn the Cartesian to spherical coordinate conversion in an unsupervised manner?
Consider the case of data points sampled from the surface of a sphere. This data lives on the spherical manifold. Assume then that this fact was not…
Angelo Rajendram
- 11
- 1
1
vote
1 answer
Why use PCA instead of manually combining feature dimensions?
Why use PCA instead of manually combining feature dimensions?
I've experimented with different datasets and plotted points using various combinations of two feature dimensions. In most cases, I can find a combination where the data points are…
Jack Miles
- 31
- 4
1
vote
0 answers
Is there any unsupervised online learning rule for classical neural networks?
In spike-based neural networks, there is a learning rule called STDP (Spike-Timing-Dependent Plasticity).
It's a completely unsupervised learning rule that works continuously when data is fed into the network.
I've been trying to find learning rules…
aarong
- 26
- 2