Unless you are strictly talking about predictive modeling, I would disagree with this. PCA just tries to represent N-dimensional observations in a k < N dimensional subspace (for a given k) such that it captures the most variation. This does not mean that any obtained component loadings refer to anything real or meaningful.
To clarify your point, we need to distinguish between pruning dimensions and projecting onto a k < N set of new orthogonal dimensions. The name of PCA does make it sound like you are selecting dimensions.
> The name of PCA does make it sound like you are selecting dimensions.
I certainly thought that!
Using somes terms from the previous comment, does this mean that the k < N subspace is not (necessarily) a subset of the N-space? Or is the subspace a subset of the data with a different coordinate system?
(Yes, I'm still trying to intuitively grasp these ideas.)
