Assessing agreement on classification tasks: the kappa statistic
Computational Linguistics
A hierarchical stochastic model for automatic prediction of prosodic boundary location
Computational Linguistics
| Hi-index | 0.00 |
For the implementation of the prosody prediction model, large scale annotated speech corpora have been widely applied. Reliability among transcribers, however, was too low for successful learning of an automatic prosodic prediction. This paper reveals our observations on performance deterioration of the learning model due to inconsistent tagging of prosodic breaks in the established corpora. Then, we suggest a method for consistent prosodic labeling among multiple transcribers. As a result, we obtain a corpus with consistent annotation of prosodic breaks. The estimated pairwise agreement of annotation of the main corpus is between 0.7477 and 0.7916, and the value of K is between 0.7057 and 0.7569. Considering the estimated K, annotation of the main corpus has reliable consistency among multiple transcribers.