An Experimental Comparison of Range Image Segmentation Algorithms
IEEE Transactions on Pattern Analysis and Machine Intelligence
Empirical Performance Evaluation of Graphics Recognition Systems
IEEE Transactions on Pattern Analysis and Machine Intelligence
Fast Similarity Search in the Presence of Noise, Scaling, and Translation in Time-Series Databases
VLDB '95 Proceedings of the 21th International Conference on Very Large Data Bases
On the need for time series data mining benchmarks: a survey and empirical demonstration
Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
Wearable sensing to annotate meeting recordings
Personal and Ubiquitous Computing
Pattern Classification (2nd Edition)
Pattern Classification (2nd Edition)
A Ubiquitous Computing environment for aircraft maintenance
Proceedings of the 2004 ACM symposium on Applied computing
Continuous Recognition of Arm Activities With Body-Worn Inertial Sensors
ISWC '04 Proceedings of the Eighth International Symposium on Wearable Computers
A novel error measure for the evaluation of video indexing systems
ICASSP '00 Proceedings of the Acoustics, Speech, and Signal Processing, 2000. on IEEE International Conference - Volume 04
A hybrid discriminative/generative approach for modeling human activities
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Where am i: recognizing on-body positions of wearable sensors
LoCA'05 Proceedings of the First international conference on Location- and Context-Awareness
Recognition of dietary activity events using on-body sensors
Artificial Intelligence in Medicine
Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography
Pervasive '08 Proceedings of the 6th International Conference on Pervasive Computing
Eye movement analysis for activity recognition
Proceedings of the 11th international conference on Ubiquitous computing
Performance metrics for activity recognition
ACM Transactions on Intelligent Systems and Technology (TIST)
Generic performance metrics for continuous activity recognition
KI'11 Proceedings of the 34th Annual German conference on Advances in artificial intelligence
The User's Touch: A Design Requirement for Smart Spaces
International Journal of Advanced Pervasive and Ubiquitous Computing
Accelerometer-based transportation mode detection on smartphones
Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems
Hi-index | 0.01 |
Evaluating the performance of a continuous activity recognition system can be a challenging problem. To-date there is no widely accepted standard for dealing with this, and in general methods and measures are adapted from related fields such as speech and vision. Much of the problem stems from the often imprecise and ambiguous nature of the real-world events that an activity recognition system has to deal with. A recognised event might have variable duration, or be shifted in time from the corresponding real-world event. Equally it might be broken up into smaller pieces, or joined together to form larger events. Most evaluation attempts tend to smooth over these issues, using “fuzzy” boundaries, or some other parameter based error decision, so as to make possible the use of standard performance measures (such as insertions and deletions.) However, we argue that reducing the various facets of a activity system into limited error categories – that were originally intended for different problem domains – can be overly restrictive. In this paper we attempt to identify and characterise the errors typical to continuous activity recognition, and develop a method for quantifying them in an unambiguous manner. By way of an initial investigation, we apply the method to an example taken from previous work, and discuss the advantages that this provides over two of the most commonly used methods.