Using social psychology to motivate contributions to online communities
CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
Controlled experiments on the web: survey and practical guide
Data Mining and Knowledge Discovery
Feed me: motivating newcomer contribution in social network sites
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Seven pitfalls to avoid when running controlled experiments on the web
Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining
A contextual-bandit approach to personalized news article recommendation
Proceedings of the 19th international conference on World wide web
Overlapping experiment infrastructure: more, better, faster experimentation
Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining
A modern Bayesian look at the multi-armed bandit
Applied Stochastic Models in Business and Industry
Proceedings of the 20th international conference on World wide web
How effective is targeted advertising?
Proceedings of the 21st international conference on World Wide Web
The role of social networks in information diffusion
Proceedings of the 21st international conference on World Wide Web
Social influence in social advertising: evidence from field experiments
Proceedings of the 13th ACM Conference on Electronic Commerce
Trustworthy online controlled experiments: five puzzling outcomes explained
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
All the news that's fit to read: a study of social annotations for news reading
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Selection effects in online sharing: consequences for peer adoption
Proceedings of the fourteenth ACM conference on Electronic commerce
Graph cluster randomization: network exposure to multiple universes
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Uncertainty in online experiments with dependent data: an evaluation of bootstrap methods
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Hi-index | 0.00 |
Online experiments are widely used to compare specific design alternatives, but they can also be used to produce generalizable knowledge and inform strategic decision making. Doing so often requires sophisticated experimental designs, iterative refinement, and careful logging and analysis. Few tools exist that support these needs. We thus introduce a language for online field experiments called PlanOut. PlanOut separates experimental design from application code, allowing the experimenter to concisely describe experimental designs, whether common "A/B tests" and factorial designs, or more complex designs involving conditional logic or multiple experimental units. These latter designs are often useful for understanding causal mechanisms involved in user behaviors. We demonstrate how experiments from the literature can be implemented in PlanOut, and describe two large field experiments conducted on Facebook with PlanOut. For common scenarios in which experiments are run iteratively and in parallel, we introduce a namespaced management system that encourages sound experimental practice.