Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces

Authors:
Aapo Hyvärinen;Patrik Hoyer
Affiliations:
Helsinki University of Technology, Laboratory of Computer and Information Science, FIN-02015 HUT, Finland;Helsinki University of Technology, Laboratory of Computer and Information Science, FIN-02015 HUT, Finland
Venue:
Neural Computation
Year:
2000

Citing 8
Cited 97

Learning invariance from transformation sequences

Neural Computation
What is the goal of sensory coding?

Neural Computation
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
A fast fixed-point algorithm for independent component analysis

Neural Computation
Independent component analysis by general nonlinear Hebbian-like learning rules

Signal Processing - Special issue on neural networks
Self-Organizing Maps

Self-Organizing Maps
Equivariant adaptive source separation

IEEE Transactions on Signal Processing
A class of neural networks for independent component analysis

IEEE Transactions on Neural Networks

Ensemble of Independent Factor Analyzers with Application to Natural Image Analysis

Neural Processing Letters
Topography as a property of the natural sensory world

Natural Computing: an international journal
Learning generative models of natural images

Neural Networks
Estimating Overcomplete Independent Component Bases for Image Windows

Journal of Mathematical Imaging and Vision
Separation of independent components from data mixed by several mixing matrices

Signal Processing - Signal processing with heavy-tailed models
Independent Variable Group Analysis

ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Learning Multiple Feature Representations from Natural Image Sequences

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Applying Slow Feature Analysis to Image Sequences Yields a Rich Repertoire of Complex Cell Properties

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Extracting Slow Subspaces from Natural Videos Leads to Complex Cells

ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Topographic ICA as a Model of Natural Image Statistics

BMVC '00 Proceedings of the First IEEE International Workshop on Biologically Motivated Computer Vision
Self-Organization of Orientation Maps, Lateral Connections, and Dynamic Receptive Fields in the Primary Visual Cortex

ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Learning optimized features for hierarchical models of invariant object recognition

Neural Computation
Beyond independent components: trees and clusters

The Journal of Machine Learning Research
Three easy ways for separating nonlinear mixtures?

Signal Processing - Special issue on independent components analysis and beyond
Blind separation of sources that have spatiotemporal variance dependencies

Signal Processing - Special issue on independent components analysis and beyond
Hierarchical models of variance sources

Signal Processing - Special issue on independent components analysis and beyond
Uniqueness of complex and multidimensional independent component analysis

Signal Processing
Beyond independent components: trees and clusters

The Journal of Machine Learning Research
A New Image Representation Algorithm Inspired by Image Submodality Models, Redundancy Reduction, and Learning in Biological Vision

IEEE Transactions on Pattern Analysis and Machine Intelligence
Feature Extraction Using Independent Components of Each Category

Neural Processing Letters
A unifying model for blind separation of independent sources

Signal Processing
Independent subspace analysis using geodesic spanning trees

ICML '05 Proceedings of the 22nd international conference on Machine learning
Email Surveillance Using Non-negative Matrix Factorization

Computational & Mathematical Organization Theory
An Adaptive Method for Subband Decomposition ICA

Neural Computation
A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals

Neural Computation
Topographic Independent Component Analysis

Neural Computation
Document clustering using nonnegative matrix factorization

Information Processing and Management: an International Journal
Neurons with Two Sites of Synaptic Integration Learn Invariant Representations

Neural Computation
On the analysis and interpretation of inhomogeneous quadratic forms as receptive fields

Neural Computation
Soft Mixer Assignment in a Hierarchical Generative Model of Natural Scene Statistics

Neural Computation
Rapid and Brief Communication: Subspace independent component analysis using vector kurtosis

Pattern Recognition
A unifying view of Wiener and Volterra theory and polynomial kernel regression

Neural Computation
Learning sensory representations with intrinsic plasticity

Neurocomputing
Linear multilayer ICA generating hierarchical edge detectors

Neural Computation
Fast fixed-point independent vector analysis algorithms for convolutive blind source separation

Signal Processing
Joint low-rank approximation for extracting non-Gaussian subspaces

Signal Processing
Topographic independent component analysis based on fractal theory and morphology applied to texture segmentation

Signal Processing
Building Blocks for Variational Bayesian Learning of Latent Variable Models

The Journal of Machine Learning Research
Undercomplete Blind Subspace Deconvolution

The Journal of Machine Learning Research
Capturing nonlinear dependencies in natural images using ICA and mixture of Laplacian distribution

Neurocomputing
Clustering of dependent components: a new paradigm for fMRI signal detection

EURASIP Journal on Applied Signal Processing
A survey of architecture and function of the primary visual cortex (V1)

EURASIP Journal on Applied Signal Processing
Multiplicative updates for non-negative projections

Neurocomputing
Extending ICA for finding jointly dependent components from two related data sets

Neurocomputing
2008 Special Issue: Principal whitened gradient for information geometry

Neural Networks
Independent subspace analysis can cope with the 'curse of dimensionality'

Acta Cybernetica
Overcomplete topographic independent component analysis

Neurocomputing
Hierarchical kernel stick-breaking process for multi-task image analysis

Proceedings of the 25th international conference on Machine learning
Estimation of causal effects using linear non-Gaussian causal models with hidden variables

International Journal of Approximate Reasoning
Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Integrating contrast invariance into a model for cortical orientation map formation

Neurocomputing
Sparse Coding Neural Gas: Learning of overcomplete data representations

Neurocomputing
Hierarchical Extraction of Independent Subspaces of Unknown Dimensions

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Non-independent BSS: A Model for Evoked MEG Signals with Controllable Dependencies

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Topographic Independent Component Analysis Based on Fractal and Morphology Applied to Texture Segmentation

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Learning Natural Image Structure with a Horizontal Product Model

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Dependent component analysis for blind restoration of images degraded by turbulent atmosphere

Neurocomputing
Nonlinear extraction of independent components of natural images using radial gaussianization

Neural Computation
Motion Synthesis Using Style-Editable Inverse Kinematics

IVA '09 Proceedings of the 9th International Conference on Intelligent Virtual Agents
Generating Self-organized Saliency Map Based on Color and Motion

ICONIP '09 Proceedings of the 16th International Conference on Neural Information Processing: Part II
Controlled complete ARMA independent process analysis

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Document clustering using nonnegative matrix factorization

Information Processing and Management: an International Journal
Letters: Nonnegative independent component analysis based on minimizing mutual information technique

Neurocomputing
Sparse coding with invariance constraints

ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing
Novel nonGaussianity measure based BSS algorithm for dependent signals

APWeb/WAIM'07 Proceedings of the joint 9th Asia-Pacific web and 8th international conference on web-age information management conference on Advances in data and web management
Modeling and estimation of dependent subspaces with non-radially symmetric and skewed densities

ICA'07 Proceedings of the 7th international conference on Independent component analysis and signal separation
Perception of transformation-invariance in the visual pathway

ICA'07 Proceedings of the 7th international conference on Independent component analysis and signal separation
Bayesian estimation of overcomplete independent feature subspaces for natural images

ICA'07 Proceedings of the 7th international conference on Independent component analysis and signal separation
Post nonlinear independent subspace analysis

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
A two-layer ICA-like model estimated by score matching

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Unifying perceptual and behavioral learning with a correlative subspace learning rule

Neurocomputing
Independent vector analysis for source separation using a mixture of gaussians prior

Neural Computation
A stylistic human motion editing system based on a subspace motion model

International Journal of Computer Applications in Technology
A two-layer model of natural stimuli estimated with score matching

Neural Computation
Viewpoint-invariant face recognition based on view-based representation

ICIC'06 Proceedings of the 2006 international conference on Intelligent computing: Part II
Lp-Nested Symmetric Distributions

The Journal of Machine Learning Research
Plaque classification using sparse features of IVUS RF signal for the diagnosis of arteriosclerosis

ACACOS'11 Proceedings of the 10th WSEAS international conference on Applied computer and applied computational science
Independent subspace analysis using k-nearest neighborhood distances

ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II
Complex-valued independent component analysis of natural images

ICANN'11 Proceedings of the 21st international conference on Artificial neural networks - Volume Part II
A review: relationship between response properties of visual neurons and advances in nonlinear approximation theory

ISNN'05 Proceedings of the Second international conference on Advances in Neural Networks - Volume Part I
Separation theorem for independent subspace analysis and its consequences

Pattern Recognition
Second-Order separation of multidimensional sources with constrained mixing system

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Riemannian optimization method on the flag manifold for independent subspace analysis

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Topographic independent component analysis of gene expression time series data

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Complex FastIVA: a robust maximum likelihood approach of MICA for convolutive BSS

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Independent subspace analysis on innovations

ECML'05 Proceedings of the 16th European conference on Machine Learning
Recent advances in subspace analysis for face recognition

SINOBIOMETRICS'04 Proceedings of the 5th Chinese conference on Advances in Biometric Person Authentication
Simple modification of oja rule limits l1-norm of weight vector and leads to sparse connectivity

Neural Computation
Learning intermediate-level representations of form and motion from natural movies

Neural Computation
Learning invariance from natural images inspired by observations in the primary visual cortex

Neural Computation
A matrix joint diagonalization approach for complex independent vector analysis

LVA/ICA'12 Proceedings of the 10th international conference on Latent Variable Analysis and Signal Separation
Contrast functions for independent subspace analysis

LVA/ICA'12 Proceedings of the 10th international conference on Latent Variable Analysis and Signal Separation
Complex events detection using data-driven concepts

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part III
Learning temporal coherent features through life-time sparsity

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Self-Organized three dimensional feature extraction of MRI and CT

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part V
Deep learning of representations: looking forward

SLSP'13 Proceedings of the First international conference on Statistical Language and Speech Processing
Improved sparse coding under the influence of perceptual attention

Neural Computation

Quantified Score

Hi-index	0.01

Visualization

Abstract

Olshausen and Field (1996) applied the principle of independence maximization by sparse coding to extract features from natural images. This leads to the emergence of oriented linear filters that have simultaneous localization in space and in frequency, thus resembling Gabor functions and simple cell receptive fields. In this article, we show that the same principle of independence maximization can explain the emergence of phase- and shift-invariant features, similar to those found in complex cells. This new kind of emergence is obtained by maximizing the independence between norms of projections on linear subspaces (instead of the independence of simple linear filter outputs). The norms of the projections on such “independent feature subspaces” then indicate the values of invariant features.