Overview of the fourth message understanding evaluation and conference
MUC4 '92 Proceedings of the 4th conference on Message understanding
Survey of the Message Understanding Conferences
HLT '93 Proceedings of the workshop on Human Language Technology
| Hi-index | 0.00 |
The evaluation of data extraction systems can be supplemented by determining performance of the systems on a representative selection of linguistic phenomena. The experiment performed as part of the MUC-3 evaluation was aimed at determining whether linguistic phenomena can be tested in isolation for state-of-the-art data extraction. Although not all of the methods of data extraction used by the participants in MUC-3 directly process linguistic phenomena, the methods are dealing with those phenomena in some manner because they are present in the input. Phenomena testing in MUC-3 is testing according to the characteristics of the messages rather than characteristics of the systems processing those messages. In order to determine the validity of phenomena testing at the current level of system performance, an experiment was run. The results of the experiment indicate that linguistic phenomena are isolatable and that performance for linguistic phenomena can be measured using the MUC-3 scoring system.