N-gram analysis based on zero-suppressed BDDs

Authors:
Ryutaro Kurai;Shin-ichi Minato;Thomas Zeugmann
Affiliations:
Hokkaido University, Sapporo, Japan;Hokkaido University, Sapporo, Japan;Hokkaido University, Sapporo, Japan
Venue:
JSAI'06 Proceedings of the 20th annual conference on New frontiers in artificial intelligence
Year:
2006

Citing 5
Cited 0

Approximate string-matching with q-grams and maximal matches

Theoretical Computer Science - Selected papers of the Combinatorial Pattern Matching School
Zero-suppressed BDDs for set manipulation in combinatorial problems

DAC '93 Proceedings of the 30th international Design Automation Conference
A new method of N-gram statistics for large number of n and automatic extraction of words and phrases from large text data of Japanese

COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
Text mining using markov chains of variable length

Proceedings of the 2005 international conference on Federation over the Web
VSOP (valued-sum-of-products) calculator for knowledge processing based on zero-suppressed BDDs

Proceedings of the 2005 international conference on Federation over the Web

Quantified Score

Hi-index	0.00

Visualization

Abstract

In the present paper, we propose a new method of n-gram analysis using ZBDDs (Zero-suppressed BDDs). ZBDDs are known as a compact representation of combinatorial item sets. Here, we newly apply the ZBDD-based techniques for efficiently handling sets of sequences. Using the algebraic operations defined over ZBDDs, such as union, intersection, difference, etc., we can execute various processings and/or analyses for large-scale sequence data. We conducted experiments for generating n-gram statistical data for given real document files. The obtained results show the potentiality of the ZBDD-based method for the sequence database analysis.