Efficient Monte Carlo computation of Fisher information matrix using prior information

Authors:
Sonjoy Das;James C. Spall;Roger Ghanem
Affiliations:
University of Southern California, Los Angeles, CA 90089, USA;Applied Physics Laboratory, The Johns Hopkins University, Laurel, MD 20723, USA;University of Southern California, Los Angeles, CA 90089, USA
Venue:
Computational Statistics & Data Analysis
Year:
2010

Citing 8
Cited 1

Elements of information theory

Elements of information theory
Iterative Weighted Least Squares Algorithms for Neural Networks Classifiers

ALT '92 Proceedings of the Third Workshop on Algorithmic Learning Theory
2008 Special Issue: Principal whitened gradient for information geometry

Neural Networks
Asymptotic Sampling Distribution for Polynomial Chaos Representation from Data: A Maximum Entropy and Fisher Information Approach

SIAM Journal on Scientific Computing
An alternative inverse Gaussian distribution

Mathematics and Computers in Simulation
CramÉr–Rao Bounds for Multiple Poles and Coefficients of Quasi-Polynomials in Colored Noise

IEEE Transactions on Signal Processing - Part I
Filtering, predictive, and smoothing Cramér-Rao bounds for discrete-time nonlinear dynamic systems

Automatica (Journal of IFAC)
Confidence interval prediction for neural network models

IEEE Transactions on Neural Networks

Computing and estimating information matrices of weak ARMA models

Computational Statistics & Data Analysis

Quantified Score

Hi-index	0.03

Visualization

Abstract

The Fisher information matrix (FIM) is a critical quantity in several aspects of mathematical modeling, including input selection and confidence region calculation. Analytical determination of the FIM in a general setting, especially in nonlinear models, may be difficult or almost impossible due to intractable modeling requirements or/and intractable high-dimensional integration. To circumvent these difficulties, a Monte Carlo simulation based technique, known as resampling algorithm, is usually recommended, in which values of the log-likelihood function or its exact stochastic gradient computed based on a set of pseudo-data vectors are used. The current work proposes an extension of this resampling algorithm in order to enhance the statistical qualities of the estimator of the FIM. This modified resampling algorithm is useful in those cases when some elements of the FIM are analytically known from prior information and the rest of the elements are unknown. The estimator of the FIM resulting from the proposed algorithm simultaneously preserves the analytically known elements and reduces variances of the estimators of the unknown elements. This is achieved by capitalizing on the information contained in the known elements.