An Evaluation of Intrinsic Dimensionality Estimators
IEEE Transactions on Pattern Analysis and Machine Intelligence
Principal component neural networks: theory and applications
Principal component neural networks: theory and applications
Limitations of nonlinear PCA as performed with generic neural networks
IEEE Transactions on Neural Networks
| Hi-index | 0.00 |
While Principal Component Analysis (PCA) is a well-known method for quantifying the redundancy between observed numerical values, it cannot effectively reduce non-linear relationships between these variables. Non-Linear Factor Analysis (NLFA) is a novel auto-associative method for data reduction, which extracts linear features to provide an easily documented encoding, while the decoding mapping is non-linear. NLFA can be implemented with standard feedforward networks if the user has control over the network configuration; many commercial packages provide the functionality needed.This type of auto-association for data reduction has not been studied or reported earlier in any depth. The (feature selection -type) encoding-decoding pair (x, y(x)) ↦ x ↦ (x, y(x)), where x and y may be vectors and y(•) is a non-linear mapping, is a natural example of linear encoding whose inverse (decoding) is non-linear. Linear feature extraction with non-linear decoding can be viewed as a generalization of feature selection - a heuristic discussion suggests this type of data reduction is effective for a wide range of practical non-linear problems. The level of effectiveness is discussed through Linearly Reducible Intrinsic Dimensionality (LRID), which is defined for continuous error-free data in a manifold; the NLFA method estimates this value from representative (typically inaccurate) discrete data.The numerically found linear encoding can be used to improve (post-process) the decoding in a manner that corresponds to a decoding network with bypass connections, and reduces the reconstruction error for the discrete data. This improved NLFA ensures that the composite function performing encoding followed by decoding, p=gof, is idempotent, p2=p, which ensures consistent results when the mapping p is viewed as "a projection to corrected values" that are in the range of the decoding g.The data reduction capacity of NLFA is between conventional PCA and the fully non-linear Kramer's NLPCA; the relationship of these three methods is discussed. Reference is provided to a separate publication, which contains numerical application examples of the basic (non-improved) NLFA.