Assessing agreement on classification tasks: the kappa statistic
Computational Linguistics
On the algorithmic implementation of multiclass kernel-based vector machines
The Journal of Machine Learning Research
Recognizing subjectivity: a case study in manual tagging
Natural Language Engineering
Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Determining the semantic orientation of terms through gloss classification
Proceedings of the 14th ACM international conference on Information and knowledge management
Using appraisal groups for sentiment analysis
Proceedings of the 14th ACM international conference on Information and knowledge management
Computational Linguistics
Thumbs up?: sentiment classification using machine learning techniques
EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Building emotion lexicon from weblog corpora
ACL '07 Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions
SemEval-2007 task 14: affective text
SemEval '07 Proceedings of the 4th International Workshop on Semantic Evaluations
Using emoticons to reduce dependency in machine learning techniques for sentiment classification
ACLstudent '05 Proceedings of the ACL Student Research Workshop
Affect analysis of text using fuzzy semantic typing
IEEE Transactions on Fuzzy Systems
A domain-independent framework for modeling emotion
Cognitive Systems Research
Hi-index | 0.00 |
In the context of Systemic Functional Linguistics, Appraisal is a theory describing the types of language utilised in communicating emotion and opinion. Robust automatic analyses of Appraisal could contribute in a number of ways to computational sentiment analysis by: distinguishing various types of evaluation, for example affect, ethics or aesthetics; discriminating between an author's opinions and the opinions of authors referenced by the author and determining the strength of evaluations. This paper reviews the typology described by Appraisal, presents a methodology for annotating Appraisal, and the use of this to annotate a corpus of book reviews. It discusses an inter-annotator agreement study, and considers instances of systematic disagreement that indicate areas in which Appraisal may be refined or clarified. Although the annotation task is difficult, there are many instances where the annotators agree; these are used to create a gold-standard corpus for future experimentation with Appraisal.