Machining planning: a model of an expert level planning process
Machining planning: a model of an expert level planning process
Automated reuse of design plans in BOGART
Artificial intelligence in engineering design (Vol. II)
Design as top-down refinement plus constraint propagation
Artificial intelligence in engineering design (Volume I)
WRIGHT: a constraint based spatial layout system
Artificial intelligence in engineering design (Volume I)
Knowledge-based system verification, validation, and testing: the evolution of a discipline
International Journal of Expert Systems - Special issue on verification and validation
Readings in knowledge acquisition and learning
Acquiring, representing, and evaluating a competence model of diagnostic strategy
Readings in knowledge acquisition and learning
What Makes for Effective Measurement?
IEEE Software
Combining naturalistic and mathematical decision aids to support product design
NDM'09 Proceedings of the 9th Bi-annual international conference on Naturalistic Decision Making
Measurement of analytical knowledge-based corporate memory and its application
Decision Support Systems
Beyond data mining; towards "idea engineering"
Proceedings of the 9th International Conference on Predictive Models in Software Engineering
| Hi-index | 0.00 |
This paper describes the QUality and Experience Metric (QUEM), a method for estimating the skill level of a knowledge-based system based on the quality of the solutions it produces. It allows one to assess how many years of experience the system would be judged to have if it were a human by providing a quantitative measure of the system's overall competence. QUEM can be viewed as a type of achievement or job-placement test administered to knowledge-based systems to help system designers determine how the system should be used and by what level of user. To apply QUEM, a set of subjects, experienced judges, and problems must be identified. The subjects should have a broad range of experience levels. Subjects and the knowledge-based system are asked to solve the problems; and judges are asked to rank order all solutions驴from worst quality to best. The data from the subjects is used to construct a skill-function relating experience to solution quality, and confidence bands showing the variability in performance. The system's quality ranking is then plugged into the skill function to produce an estimate of the system's experience level. QUEM can be used to gauge the experience level of an individual system, to compare two systems, or to compare a system to its intended users. This represents an important advance in providing quantitative measures of overall performance that can be applied to a broad range of systems.