Extracting regression rules from neural networks

Authors:
Kazumi Saito;Ryohei Nakano
Affiliations:
NTT Communication Science Laboratories, NTT Corporation, 2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan;Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya 466-8555, Japan
Venue:
Neural Networks
Year:
2002

Citing 14
Cited 15

Scientific discovery: computational explorations of the creative process

Scientific discovery: computational explorations of the creative process
Product units: a computationally powerful and biologically plausible extension to backpropagation networks

Neural Computation
Integrating quantitative and qualitative discovery in the ABACUS system

Machine learning
C4.5: programs for machine learning

C4.5: programs for machine learning
Extracting Refined Rules from Knowledge-Based Neural Networks

Machine Learning
Partial BFGS update and efficient step-length calculation for three-layer neural networks

Neural Computation
Rule extraction by successive regularization

Neural Networks
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Discovering Empirical Laws of Web Dynamics

SAINT '02 Proceedings of the 2002 Symposium on Applications and the Internet
Computational Revision of Quantitative Scientific Models

DS '01 Proceedings of the 4th International Conference on Discovery Science
Second-Order Learning Algorithm with Squared Penalty Term

Neural Computation
Law discovery using neural networks

IJCAI'97 Proceedings of the Fifteenth international joint conference on Artifical intelligence - Volume 2
The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

IEEE Transactions on Neural Networks
Neuro-fuzzy rule generation: survey in soft computing framework

IEEE Transactions on Neural Networks

Rule extraction: using neural networks or for neural networks?

Journal of Computer Science and Technology
Prediction of MHC II-binding peptides using rough set-based rule sets ensemble

Applied Intelligence
Multilogistic regression by means of evolutionary product-unit neural networks

Neural Networks
Convergence of BP algorithm for product unit neural networks with exponential weights

Neurocomputing
Evolutionary product-unit neural networks classifiers

Neurocomputing
Generating rules with predicates, terms and variables from the pruned neural networks

Neural Networks
Bidirectional Clustering of MLP Weights for Finding Nominally Conditioned Polynomials

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Knowledge-internalization process for neural-networks practitioners

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Hybrid evolutionary algorithm with product-unit neural networks for classification

IWANN'07 Proceedings of the 9th international work conference on Artificial neural networks
Memetic algorithms to product-unit neural networks for regression

IWANN'05 Proceedings of the 8th international conference on Artificial Neural Networks: computational Intelligence and Bioinspired Systems
Evolutionary product-unit neural networks for classification

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Combining the advantages of neural networks and decision trees for regression problems in a steel temperature prediction system

HAIS'12 Proceedings of the 7th international conference on Hybrid Artificial Intelligent Systems - Volume Part II
Modeling microalgal abundance with artificial neural networks: Demonstration of a heuristic 'Grey-Box' to deconvolve and quantify environmental influences

Environmental Modelling & Software
An Abductive-Reasoning Guide for Finance Practitioners

Computational Economics
On-line modeling via fuzzy support vector machines and neural networks

Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology - Recent Advances in Soft Computing: Theories and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes a new framework and method for extracting regression rules from neural networks trained with multivariate data containing both nominal and numeric variables. Each regression rule is expressed as a pair of a logical formula on the conditional part over nominal variables and a polynomial equation on the action part over numeric variables. The proposed extraction method first generates one such regression rule for each training sample, then utilizes the k-means algorithm to generate a much smaller set of rules having more general conditions, where the number of distinct polynomial equations is determined through cross-validation. Finally, this method invokes decision-tree induction to form logical formulae of nominal conditions as conditional parts of final regression rules. Experiments using four data sets show that our method works well in extracting quite accurate and interesting regression rules.