Adaptive Probabilistic Networks with Hidden Variables

Authors:
John Binder;Daphne Koller;Stuart Russell;Keiji Kanazawa
Affiliations:
Computer Science Division, University of California, Berkeley, CA 94720-1776. E-mail: binder@cs.berkeley.edu;Computer Science Department, Stanford University, Stanford, CA 94305-9010. E-mail: koller@cs.stanford.edu;Computer Science Division, University of California, Berkeley, CA 94720-1776. E-mail: russell@cs.berkeley.edu;Microsoft Corporation, One Microsoft Way, Redmond, WA 98052-6399. E-mail: keijik@microsoft.com
Venue:
Machine Learning - Special issue on learning with probabilistic representations
Year:
1997

Citing 25
Cited 94

Probabilistic reasoning in intelligent systems: networks of plausible inference

Probabilistic reasoning in intelligent systems: networks of plausible inference
Gaussian influence diagrams

Management Science
Adapting connectionist learning to Bayes networks

International Journal of Approximate Reasoning
Fundamental concepts of qualitative probabilistic networks

Artificial Intelligence
A Bayesian Method for the Induction of Probabilistic Networks from Data

Machine Learning
aHUGIN: a system creating adaptive causal probabilistic networks

UAI '92 Proceedings of the eighth conference on Uncertainty in Artificial Intelligence
Connectionist learning of belief networks

Artificial Intelligence
A practical Bayesian framework for backpropagation networks

Neural Computation
Asymmetric parallel Boltzmann machines are belief networks

Neural Computation
Knowledge-based artificial neural networks

Artificial Intelligence
Bayesian networks

Communications of the ACM
The EM algorithm for graphical association models with missing data

Computational Statistics & Data Analysis - Special issue dedicated to Toma´sˇ Havra´nek
Using hidden nodes in Bayesian networks

Artificial Intelligence
Probabilistic independence networks for hidden Markov probability models

Neural Computation
Bayesian Network Classifiers

Machine Learning - Special issue on learning with probabilistic representations
The Sample Complexity of Learning Fixed-Structure Bayesian Networks

Machine Learning - Special issue on learning with probabilistic representations
Factorial Hidden Markov Models

Machine Learning - Special issue on learning with probabilistic representations
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
A Guide to the Literature on Learning Probabilistic Networks from Data

IEEE Transactions on Knowledge and Data Engineering
Simulation Approaches to General Probabilistic Inference on Belief Networks

UAI '89 Proceedings of the Fifth Annual Conference on Uncertainty in Artificial Intelligence
Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks

UAI '89 Proceedings of the Fifth Annual Conference on Uncertainty in Artificial Intelligence
A theoretical framework for context-sensitive temporal probability model construction with application to plan projection

UAI'95 Proceedings of the Eleventh conference on Uncertainty in artificial intelligence
Learning Bayesian networks with local structure

UAI'96 Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence
On the sample complexity of learning Bayesian networks

UAI'96 Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence
Asymptotic model selection for directed networks with hidden variables*

UAI'96 Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence

Bayesian Network Classifiers

Machine Learning - Special issue on learning with probabilistic representations
Learning agents for uncertain environments (extended abstract)

COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Mixed Memory Markov Models: Decomposing Complex Stochastic Processes as Mixtures of Simpler Ones

Machine Learning
Bayesianism and Language Change

Journal of Logic, Language and Information
Parameter Learning in Object-Oriented Bayesian Networks

Annals of Mathematics and Artificial Intelligence
Towards a More Efficient Evolutionary Induction of Bayesian Networks

PPSN VII Proceedings of the 7th International Conference on Parallel Problem Solving from Nature
Local Search Methods for Learning Bayesian Networks Using a Modified Neighborhood in the Space of DAGs

IBERAMIA 2002 Proceedings of the 8th Ibero-American Conference on AI: Advances in Artificial Intelligence
Adaptive Bayes

IBERAMIA 2002 Proceedings of the 8th Ibero-American Conference on AI: Advances in Artificial Intelligence
Learning Bayesian Networks with Hidden Variables Using the Combination of EM and Evolutionary Algorithms

PAKDD '01 Proceedings of the 5th Pacific-Asia Conference on Knowledge Discovery and Data Mining
Unsupervised Learning of Probabilistic Concept Hierarchies

Machine Learning and Its Applications, Advanced Lectures
Adaptive Bayesian Logic Programs

ILP '01 Proceedings of the 11th International Conference on Inductive Logic Programming
Structural extension to logistic regression: discriminative parameter learning of belief net classifiers

Eighteenth national conference on Artificial intelligence
Fusion of domain knowledge with data for structural learning in object oriented domains

The Journal of Machine Learning Research
Identifying Markov Blankets with Decision Tree Induction

ICDM '03 Proceedings of the Third IEEE International Conference on Data Mining
Video-based event recognition: activity representation and probabilistic recognition methods

Computer Vision and Image Understanding - Special issue on event detection in video
Layered representations for learning and inferring office activity from multiple sensory channels

Computer Vision and Image Understanding - Special issue on event detection in video
Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers

Machine Learning
A Bayesian network based sequential inference for diagnosis of diseases from retinal images

Pattern Recognition Letters - Special issue: Advances in pattern recognition
A Bayesian network classifier with inverse tree structure for voxelwise magnetic resonance image analysis

Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Evolutionary online services

ICEC '05 Proceedings of the 7th international conference on Electronic commerce
Learning first-order probabilistic models with combining rules

ICML '05 Proceedings of the 22nd international conference on Machine learning
Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks

Evolutionary Computation
Attractor Dynamics in Feedforward Neural Networks

Neural Computation
Classification using Hierarchical Naïve Bayes models

Machine Learning
An adjustment model in a geometric constraint solving problem

Proceedings of the 2006 ACM symposium on Applied computing
The max-min hill-climbing Bayesian network structure learning algorithm

Machine Learning
An Architecture for Dynamical News Providers

WI-IATW '06 Proceedings of the 2006 IEEE/WIC/ACM international conference on Web Intelligence and Intelligent Agent Technology
A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests

The Journal of Machine Learning Research
Three research challenges at the intersection of machine learning, statistical induction, and systems

HOTOS'05 Proceedings of the 10th conference on Hot Topics in Operating Systems - Volume 10
Correlating instrumentation data to system states: a building block for automated diagnosis and control

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Bayesian network learning algorithms using structural restrictions

International Journal of Approximate Reasoning
Parameter learning for relational Bayesian networks

Proceedings of the 24th international conference on Machine learning
Event detection using "variable module graphs" for home care applications

EURASIP Journal on Applied Signal Processing
Interactive dynamic production by genetic algorithms

AIAP'07 Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: artificial intelligence and applications
A hybrid Bayesian network learning method for constructing gene networks

Computational Biology and Chemistry
A model for parameter setting based on Bayesian networks

Engineering Applications of Artificial Intelligence
Quantifying the uncertainty of a belief net response: Bayesian error-bars for belief net inference

Artificial Intelligence
A Recursive Method for Structural Learning of Directed Acyclic Graphs

The Journal of Machine Learning Research
A Context-Aware Movie Preference Model Using a Bayesian Network for Recommendation and Promotion

UM '07 Proceedings of the 11th international conference on User Modeling
Bayesian Substructure Learning - Approximate Learning of Very Large Network Structures

ECML '07 Proceedings of the 18th European conference on Machine Learning
Looking Ahead to Select Tutorial Actions: A Decision-Theoretic Approach

International Journal of Artificial Intelligence in Education
Using Markov Blankets for Causal Structure Learning

The Journal of Machine Learning Research
ADAPTIVE MACHINE LEARNING IN DELAYED FEEDBACK DOMAINS BY SELECTIVE RELEARNING

Applied Artificial Intelligence
Learning Causal Bayesian Networks from Incomplete Observational Data and Interventions

ECSQARU '07 Proceedings of the 9th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Semantic event representation and recognition using syntactic attribute graph grammar

Pattern Recognition Letters
Structured Learning of Component Dependencies in AmI Systems

WI-IAT '08 Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 02
Global and componentwise extrapolations for accelerating training of Bayesian networks and conditional random fields

Data Mining and Knowledge Discovery
Learning Bayesian network parameters under incomplete data with domain knowledge

Pattern Recognition
Approximation Methods for Efficient Learning of Bayesian Networks

Proceedings of the 2008 conference on Approximation Methods for Efficient Learning of Bayesian Networks
An Inductive Logic Programming Approach to Statistical Relational Learning

Proceedings of the 2005 conference on An Inductive Logic Programming Approach to Statistical Relational Learning
Bayesian learning of Bayesian networks with informative priors

Annals of Mathematics and Artificial Intelligence
Learning first-order probabilistic models with combining rules

Annals of Mathematics and Artificial Intelligence
Bayesian network structure learning using cooperative coevolution

Proceedings of the 11th Annual conference on Genetic and evolutionary computation
Searching for Bayesian network structures in the space of restricted acyclic partially directed graphs

Journal of Artificial Intelligence Research
Using a local discovery ant algorithm for Bayesian network structure learning

IEEE Transactions on Evolutionary Computation
Inference in hybrid Bayesian networks with mixtures of truncated exponentials

International Journal of Approximate Reasoning
Organization reliability modeling of ship oil spill emergency management based on Bayesian network

WiCOM'09 Proceedings of the 5th International Conference on Wireless communications, networking and mobile computing
Bayesian Network Structure Learning by Recursive Autonomy Identification

The Journal of Machine Learning Research
Optimal Search on Clustered Structural Constraint for Learning Bayesian Network Structure

The Journal of Machine Learning Research
Revision of first-order Bayesian classifiers

ILP'02 Proceedings of the 12th international conference on Inductive logic programming
An evolutionary algorithm for adaptive online services in dynamic environment

Evo'08 Proceedings of the 2008 conference on Applications of evolutionary computing
Basic principles of learning Bayesian logic programs

Probabilistic inductive logic programming
Generalized loopy 2U: A new algorithm for approximate inference in credal networks

International Journal of Approximate Reasoning
Research Note---Toward a Causal Interpretation from Observational Data: A New Bayesian Networks Method for Structural Models with Latent Variables

Information Systems Research
An efficient causal discovery algorithm for linear models

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
Learning an L1-regularized Gaussian Bayesian network in the equivalence class space

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Comparative analysis of search and score metaheuristics for Bayesian network structure learning using node juxtaposition distributions

PPSN'10 Proceedings of the 11th international conference on Parallel problem solving from nature: Part I
Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

Data Mining and Knowledge Discovery
Learning the behavior model of a robot

Autonomous Robots
A cooperative coevolutionary genetic algorithm for learning bayesian network structures

Proceedings of the 13th annual conference on Genetic and evolutionary computation
A variational approximation for Bayesian networks with discrete and continuous latent variables

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Accelerating EM: an empirical study

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Gaussian process networks

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Bayesian error-bars for belief net inference

UAI'01 Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
The Bayesian structural EM algorithm

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Learning the structure of dynamic probabilistic networks

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Update rules for parameter estimation in Bayesian networks

UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Learning bayesian network equivalence classes from incomplete data

DS'06 Proceedings of the 9th international conference on Discovery Science
An architecture for evolutionary adaptive web systems

WINE'05 Proceedings of the First international conference on Internet and Network Economics
MCMC learning of bayesian network models by markov blanket decomposition

ECML'05 Proceedings of the 16th European conference on Machine Learning
Constrained score+(local)search methods for learning bayesian networks

ECSQARU'05 Proceedings of the 8th European conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
On the use of restrictions for learning bayesian networks

ECSQARU'05 Proceedings of the 8th European conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Review: learning bayesian networks: Approaches and issues

The Knowledge Engineering Review
The IMAP hybrid method for learning gaussian bayes nets

AI'10 Proceedings of the 23rd Canadian conference on Advances in Artificial Intelligence
Learning Causal Relations in Multivariate Time Series Data

ACM Transactions on Intelligent Systems and Technology (TIST)
Towards integrative causal analysis of heterogeneous data sets and studies

The Journal of Machine Learning Research
Learning bayesian networks from Markov random fields: An efficient algorithm for linear models

ACM Transactions on Knowledge Discovery from Data (TKDD)
Score-based methods for learning Markov boundaries by searching in constrained spaces

Data Mining and Knowledge Discovery
Scaling up the Greedy Equivalence Search algorithm by constraining the search space of equivalence classes

International Journal of Approximate Reasoning
A memetic approach to bayesian network structure learning

EvoApplications'13 Proceedings of the 16th European conference on Applications of Evolutionary Computation
Identifying significant edges in graphical models of molecular networks

Artificial Intelligence in Medicine
Inference for a new probabilistic constraint logic

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
A survey on latent tree models and applications

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	0.01

Visualization

Abstract

Probabilistic networks (also known as Bayesian belief networks)allow a compact description of complex stochastic relationships amongseveral random variables. They are used widely for uncertain reasoning inartificial intelligence. In this paper, we investigate the problem oflearning probabilistic networks with known structure and hidden variables.This is an important problem, because structure is much easier to elicitfrom experts than numbers, and the world is rarely fully observable. Wepresent a gradient-based algorithm and show that the gradient can becomputed locally, using information that is available as a byproduct ofstandard inference algorithms for probabilistic networks. Our experimentalresults demonstrate that using prior knowledge about the structure, evenwith hidden variables, can significantly improve the learning rate ofprobabilistic networks. We extend the method to networks in which theconditional probability tables are described using a small number ofparameters. Examples include noisy-OR nodes and dynamic probabilisticnetworks. We show how this additional structure can be exploited by ouralgorithm to speed up the learning even further. We also outline anextension to hybrid networks, in which some of the nodestake on values in a continuous domain.