Measures of mutual and causal dependence between two time series
IEEE Transactions on Information Theory
Elements of information theory
Elements of information theory
Graphical Models: Foundations of Neural Computation
Graphical Models: Foundations of Neural Computation
Networks: An Introduction
Journal of Computational Neuroscience
Transfer entropy--a model-free measure of effective connectivity for the neurosciences
Journal of Computational Neuroscience
Hi-index | 0.07 |
This study aims at providing the definitive links between Massey and Kramer's directed information theory and Granger causality graphs, recently formalized by Eichler. This naturally leads to consider two concepts of causality that can occur in physical systems: dynamical causality and instantaneous coupling. Although it is well accepted that the former is assessed by conditional transfer entropy, the latter is often overlooked, even if it was clearly introduced and understood in seminal studies. In the bivariate case, we show that directed information is decomposed into the sum of transfer entropy and instantaneous information exchange. In the multivariate case, encountered for conditional graph modeling, such a decomposition does not hold anymore. We provide a new insight into the problem of instantaneous coupling and show how it affects the estimated structure of a graphical model that should provide a sparse representation of a complex system. This result is discussed and analyzed for the inference of causality graphs. Two synthetic examples are developed to illustrate the theoretical concepts. Practical issues are also briefly discussed on these examples and an extension of Leonenko's k-nearest neighbors based entropy estimators is used to derive a nonparametric estimator of directed information.