Classification using conditional probabilities and Shannon's definition of information

• Authors: Andrew Borden
• Venue: Proceedings of the 2007 International Lisp Conference
• Year: 2007

Our problem is to build a maximally efficient Bayesian classifier when each parameter has a different cost and provides a different amount of information toward the solution. This is an extremely computationally expensive problem. We accept a sub-optimal, although demonstrably good, solution based on Shannon's definition of Information and Uncertainty. Our solution scales up well and provides powerful diagnostics with no extra work. Our program has been used in Military Command and Control and in developing maintenance diagnostics for a very complex military electronic system. The elements of the solution to the problem are naturally computed recursively, so Lisp implementation is an effective approach.