Prioritizing Information for the Discovery of Phenomena
Journal of Intelligent Information Systems
Detecting anomalous records in categorical datasets
Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Anomaly pattern detection in categorical datasets
Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Research on hidden Markov model for system call anomaly detection
PAISI'07 Proceedings of the 2007 Pacific Asia conference on Intelligence and security informatics
An anomaly intrusion detection approach using cellular neural networks
ISCIS'06 Proceedings of the 21st international conference on Computer and Information Sciences
| Hi-index | 0.00 |
This paper examines the problem of prioritizing actions under uncertainty. Our motivating applications come from the domain of data mining. Data mining problems present the user with a huge collection of individual items (e.g., abstracts, medical histories, and computer users' command histories) and require that these items be prioritized according to which should be pursued thoroughly. More precisely, each data item is assumed to be generated by one of two processes: A large majority of the data comes from a common, mundane process and a very small fraction comes from a rare, phenomenon process. The problem is to rank the information so as to optimally direct the user in his or her pursuit of the data items that were generated by the phenomenon process. Our previous work has developed the theoretical foundations of the information prioritization problem. The current paper summarizes these foundations, derives new theoretical results, and details initial experimental results of a prioritization system based on the theory. We focus here on feature selection techniques and the method of model surrogates, each tailored to the classes of prioritization applications of greatest current interest. Our results demonstrate the effectiveness of the techniques and motivate further research to improve the existing system