Principles of Information Systems
Principles of Information Systems
Variations in use of meta tag descriptions by web pages in different languages
Information Processing and Management: an International Journal
Characteristics of scientific web publications: preliminary data gathering and analysis
Journal of the American Society for Information Science and Technology - Special issue: Webometrics
Does topic metadata help with Web search?
Journal of the American Society for Information Science and Technology
A large-scale study of robots.txt
Proceedings of the 16th international conference on World Wide Web
ICWE '9 Proceedings of the 9th International Conference on Web Engineering
Journal of Web Engineering
Hi-index | 0.00 |
Sixty corporate websites selected from the Fortune Global 500 companies were examined in 2000 and again in 2001 to see if they provided support for automatic indexing. In particular, use of the robots.txt and Meta tags for "keywords" and "description" was examined. Slightly fewer than half of the sites provided one or both of these aids. Among sites providing indexing aids there was a clear under-representation of Asian sites. Nearly 80% of the sites used Java, suggesting a reasonable level of technical sophistication among website creators. About one-third of the sites used cookies, raising the possibility that repeat visitors might find the navigation of the site customized to their needs. Overall an increase in the use of indexing aids, especially Meta tags, represents one way in which web robots could index sites more quickly and thus improve overall index coverage of the web.