An evaluation of retrieval effectiveness for a full-text document-retrieval system
Communications of the ACM
The cost structure of sensemaking
CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
Modern Information Retrieval
A symbiotic theory formation system
A symbiotic theory formation system
Introduction to Information Retrieval
Introduction to Information Retrieval
The centrality of user modeling to high recall with high precision search
SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Evaluation of information retrieval for E-discovery
Artificial Intelligence and Law
Automation of legal sensemaking in e-discovery
Artificial Intelligence and Law
| Hi-index | 0.00 |
Treating the information retrieval (IR) task as one of classification has been shown to be the most effective way to achieve high performance. In real-world Systems, a human is the ultimate determinant of relevance and must be integrated symbiotically into the control structures. We report on a hybrid, Human-Assisted Computer Classification system that opportunistically pairs processes of Active Learning and User Modeling to produce a high-Q computational engine. Top-down human goals are impedance-matched with bottom-up corpus analysis utilizing critical control loops. The System contributions of humans and machines as 'Proxy,' 'Assessor,' and 'Classifier' elements are blended through inter-related 'Model,' 'Match,' and 'Measure' processes (M3) to achieve consistently high precision IR with high recall. We report results for over a dozen topics, with confirmation of internal measures from topic 103 of the 2008 TREC legal track's interactive task.