Ontology-based realtime activity monitoring using beam search
ICVS'11 Proceedings of the 8th international conference on Computer vision systems
Probabilistic event calculus based on Markov logic networks
RuleML'11 Proceedings of the 5th international conference on Rule-based modeling and computing on the semantic web
Event processing under uncertainty
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems
Combining per-frame and per-track cues for multi-person action recognition
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part I
Modeling complex temporal composition of actionlets for activity prediction
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part I
International Journal of Computer Vision
Assessing team strategy using spatiotemporal data
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Image and Vision Computing
A Markov logic framework for recognizing complex events from multimodal data
Proceedings of the 15th ACM on International conference on multimodal interaction
Efficient extraction and representation of spatial information from video data
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Complex event processing over distributed probabilistic event streams
Computers & Mathematics with Applications
Hi-index | 0.00 |
We present a framework for the automatic recognition of complex multi-agent events in settings where structure is imposed by rules that agents must follow while performing activities. Given semantic spatio-temporal descriptions of what generally happens (i.e., rules, event descriptions, physical constraints), and based on video analysis, we determine the events that occurred. Knowledge about spatio-temporal structure is encoded using first-order logic using an approach based on Allen's Interval Logic, and robustness to low-level observation uncertainty is provided by Markov Logic Networks (MLN). Our main contribution is that we integrate interval-based temporal reasoning with probabilistic logical inference, relying on an efficient bottom-up grounding scheme to avoid combinatorial explosion. Applied to one-on-one basketball, our framework detects and tracks players, their hands and feet, and the ball, generates event observations from the resulting trajectories, and performs probabilistic logical inference to determine the most consistent sequence of events. We demonstrate our approach on 1hr (100,000 frames) of outdoor videos.