A trigram hidden Markov model for metadata extraction from heterogeneous references

  • Authors:

  • Bolanle Ojokoh;Ming Zhang;Jian Tang

  • Affiliations:

  • School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, PR China and Department of Computer Science, Federal University of Technology, P.M.B. 704 Akure, Nigeria;School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, PR China;School of Electronic Engineering and Computer Science, Peking University, Beijing 100871, PR China

  • Venue:
  • Information Sciences: an International Journal
  • Year:
  • 2011

Quantified Score

Hi-index 0.08

Visualization

Abstract

Our objective was to explore an efficient and accurate extraction of metadata such as author, title and institution from heterogeneous references, using hidden Markov models (HMMs). The major contributions of the research were the (i) development of a trigram, full second order hidden Markov model with more priority to words emitted in transitions to the same state, with a corresponding new Viterbi algorithm (ii) introduction of a new smoothing technique for transition probabilities and (iii) proposal of a modification of back-off shrinkage technique for emission probabilities. The effect of the size of data set on the training procedure was also measured. Comparisons were made with other related works and the model was evaluated with three different data sets. The results showed overall accuracy, precision, recall and F1 measure of over 95% suggesting that the method outperforms other related methods in the task of metadata extraction from references.