The changing face of web search: algorithms, auctions and advertising

• Authors: Prabhakar Raghavan
• Venue: Proceedings of the thirty-eighth annual ACM symposium on Theory of computing
• Year: 2006

Web search has come to dominate our consciousness as a convenience we take for granted, as a medium for connecting advertisers and buyers, and as a fast-growing revenue source for the companies that provide this service. Following a brief overview of the state of the art and how we got there, this talk covers a spectrum of technical challenges arising in web search.This lecture will begin with an overview of the social, economic and historical challenges underlying web search. Understanding the basic background is a useful prerequisite for deep technical work in this area. Following this, we will cover three vignettes, whose goal is to expose significant research areas rather than to present definitive results.The first deals with an emerging area variously referred to as Human Computation, Social Computation or Social Media. The idea is to solve difficult problems in artificial intelligence (such as image recognition) not through direct computation, but by exploiting the wisdom of crowds on the web. In the simplest form, an incentive mechanism is devised whereby many web users label images descriptively. These labels are then used for image retrieval. This immediately raises several foundational questions. What incentive mechanisms lead to high-quality labels? Given the inevitability of misleading labels (spam), how does one filter out good labels? Since the participants in such a system are likely to be connected in various social networks, how does one propagate trust and reputation in these networks to obtain reliable judges and thereby judgments.The second vignette centers around optimization and marketplace design for advertisements on the internet. We first outline how the presentation of brand advertisement on the internet leads to stochastic programming problems - in turn leading to novel issues in the design of futures contracts. We then turn to a problem more heavily studied in the theoretical computer science literature: the auction design and pricing of advertisement on keyword search results. Beginning with the classic Vickrey auction, known to be a truthful mechanism for single-item, sealed-bid auctions, we point out how sponsored search advertisements depart from this simple setting. We review the current state of the art here and mention several problems that remain open.The final vignette is based on the paper with Kleinberg. We formulate a model for query incentive networks, motivated by users seeking information or services that pose queries, together with incentives for answering them. This type of information-seeking process can be formulated as a game among the nodes in the network, and this game has a natural Nash equilibrium. How much incentive is needed in order to achieve a reasonable probability of obtaining an answer to a query? We study the size of query incentives as a function both of the rarity of the answer and the structure of the underlying network. This leads to natural questions related to strategic behavior in branching processes. Whereas the classically studied criticality of branching processes is centered around the region where the branching parameter is 1, we show in contrast that strategic interaction in incentive propagation exhibits critical behavior when the branching parameter is 2.