Flexible document categorisation
AIKED'05 Proceedings of the 4th WSEAS International Conference on Artificial Intelligence, Knowledge Engineering Data Bases
Information Processing and Management: an International Journal
Cybergenre: automatic identification of home pages on the web
Journal of Web Engineering
Hi-index | 0.00 |
Genre characterizes text differently than the usual subject or prepositional content that has been the focus of most information retrieval and classification research. We developed a newmethod for automatic genre classification that is based on statistically selected features obtained from both subject-classified and genre-classified training data. The main idea of the genre classification method is to calculate the weight of a feature for a genre class by using itsfrequency statistics for subject classifications. Having observed that the deviation formula anddiscrimination formula using document frequency ratios work as expected, we went on to study the roles of various types of features such as content-bearing words, function words,morphemes, and punctuations marks. In the first part of this paper, we present some of ourfindings in the roles of the feature types for genre classification, with a brief discussion of the genre-based classification. The genre classes we used are those often found in Web documents: accident reportages, newspaper editorials, personal homepages, product reviews,product specifications, research articles, and Q&A's. The second part of the paper addresses the issue of how text genres help classifying documents based on the subject content of documents. This is a corollary to our original hypothesis that subject classification would help identifying the genre class of a document automatically. Our experimental work shows that while subject classes clearly help improving the genre-based classification, it is not clear whether using the genre class information for documents in the same way helps subject-based classification. However, we found that training a subject classifier with a set of documents belonging to a particular genre class improves subject-based classification.