Testing web sites: five users is nowhere near enough
CHI '01 Extended Abstracts on Human Factors in Computing Systems
Web credibility research: a method for online experiments and early study results
CHI '01 Extended Abstracts on Human Factors in Computing Systems
He says, she says: conflict and coordination in Wikipedia
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
What happened to remote usability testing?: an empirical study of three methods
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
OCSC'07 Proceedings of the 2nd international conference on Online communities and social computing
Harnessing the wisdom of crowds in wikipedia: quality through coordination
Proceedings of the 2008 ACM conference on Computer supported cooperative work
Can you ever trust a wiki?: impacting perceived trustworthiness in wikipedia
Proceedings of the 2008 ACM conference on Computer supported cooperative work
Towards a model of understanding social search
Proceedings of the 2008 ACM conference on Computer supported cooperative work
Survey and evaluation of query intent detection methods
Proceedings of the 2009 workshop on Web Search Click Data
Signpost from the masses: learning effects in an exploratory social tag search browser
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Coordination in collective intelligence: the role of team structure and task interdependence
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
What's in Wikipedia?: mapping topics and conflict using socially annotated category structure
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
With a little help from my friends: examining the impact of social annotations in sensemaking tasks
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CHI '09 Extended Abstracts on Human Factors in Computing Systems
Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
Data quality from crowdsourcing: a study of annotation selection criteria
HLT '09 Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing
Relevance criteria for e-commerce: a crowdsourcing-based experimental analysis
Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
Financial incentives and the "performance of crowds"
Proceedings of the ACM SIGKDD Workshop on Human Computation
The effect of linguistic devices in information presentation messages on comprehension and recall
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
Fraud Detection by Human Agents: A Pilot Study
EC-Web 2009 Proceedings of the 10th International Conference on E-Commerce and Web Technologies
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
A crowdsourceable QoE evaluation framework for multimedia content
MM '09 Proceedings of the 17th ACM international conference on Multimedia
Clustering and exploring search results using timeline constructions
Proceedings of the 18th ACM conference on Information and knowledge management
Interpreting written how-to instructions
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Learning by Foraging: The Impact of Social Tags on Knowledge Acquisition
EC-TEL '09 Proceedings of the 4th European Conference on Technology Enhanced Learning: Learning in the Synergy of Multiple Disciplines
Acquiring high quality non-expert knowledge from on-demand workforce
People's Web '09 Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources
Crowd translator: on building localized speech recognizers through micropayments
ACM SIGOPS Operating Systems Review
Gracefully mitigating breakdowns in robotic services
Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction
Crowdsourcing graphical perception: using mechanical turk to assess visualization design
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Enhancing directed content sharing on the web
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Characterizing debate performance via aggregated twitter sentiment
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Presenting diverse political opinions: how and how much
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Are your participants gaming the system?: screening mechanical turk workers
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Tangible interfaces for download: initial observations from users' everyday environments
CHI '10 Extended Abstracts on Human Factors in Computing Systems
Who are the crowdworkers?: shifting demographics in mechanical turk
CHI '10 Extended Abstracts on Human Factors in Computing Systems
Exploring iterative and parallel human computation processes
CHI '10 Extended Abstracts on Human Factors in Computing Systems
Modulating video credibility via visualization of quality evaluations
Proceedings of the 4th workshop on Information credibility
No Code Required: Giving Users Tools to Transform the Web
No Code Required: Giving Users Tools to Transform the Web
Financial incentives and the "performance of crowds"
ACM SIGKDD Explorations Newsletter
Collaborative identification and annotation of government deep web resources: a hybrid approach
Proceedings of the 21st ACM conference on Hypertext and hypermedia
CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones
Proceedings of the 8th international conference on Mobile systems, applications, and services
Crowdsourcing the assembly of concept hierarchies
Proceedings of the 10th annual joint conference on Digital libraries
Putting the crowd to work in a knowledge-based factory
Advanced Engineering Informatics
Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization
The anatomy of a large-scale human computation engine
Proceedings of the ACM SIGKDD Workshop on Human Computation
Sellers' problems in human computation markets
Proceedings of the ACM SIGKDD Workshop on Human Computation
Exploring iterative and parallel human computation processes
Proceedings of the ACM SIGKDD Workshop on Human Computation
Exploring the use of crowdsourcing to support empirical studies in software engineering
Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement
LilyPad in the wild: how hardware's long tail is supporting new engineering and design communities
Proceedings of the 8th ACM Conference on Designing Interactive Systems
Using Crowdsourcing and Active Learning to Track Sentiment in Online Media
Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence
Three challenges in data mining
Frontiers of Computer Science in China
An elaborated model of social search
Information Processing and Management: an International Journal
Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Adjunct
Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Adjunct
TurKit: human computation algorithms on mechanical turk
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Soylent: a word processor with a crowd inside
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
VizWiz: nearly real-time answers to visual questions
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology
Consensus versus expertise: a case study of word alignment with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Annotating large email datasets for named entity recognition with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Collecting image annotations using Amazon's Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Evaluation of commonsense knowledge with Mechanical Turk
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Preliminary experience with Amazon's Mechanical Turk for annotating medical named entities
CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
PackPlay: mining semantic data in collaborative games
LAW IV '10 Proceedings of the Fourth Linguistic Annotation Workshop
Crowdsourcing human-based computation
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Computer Speech and Language
Using cultures of participation to change behaviors
Proceedings of the 16th ACM international conference on Supporting group work
Challenges and experiences in deploying enterprise crowdsourcing service
ICWE'10 Proceedings of the 10th international conference on Web engineering
Collaboration on Social Network Sites: Amateurs, Professionals and Celebrities
Computer Supported Cooperative Work
Towards the web of concepts: extracting concepts from large datasets
Proceedings of the VLDB Endowment
Scalable crisis relief: Crowdsourced SMS translation and categorization with Mission 4636
Proceedings of the First ACM Symposium on Computing for Development
Peoplecloud for the globally integrated enterprise
ICSOC/ServiceWave'09 Proceedings of the 2009 international conference on Service-oriented computing
Everyone's an influencer: quantifying influence on twitter
Proceedings of the fourth ACM international conference on Web search and data mining
Ranking from pairs and triplets: information quality, evaluation methods and query complexity
Proceedings of the fourth ACM international conference on Web search and data mining
Computers in Human Behavior
Human-assisted graph search: it's okay to ask questions
Proceedings of the VLDB Endowment
Designing incentives for inexpert human raters
Proceedings of the ACM 2011 conference on Computer supported cooperative work
Turkalytics: analytics for human computation
Proceedings of the 20th international conference on World wide web
A case for query by image and text content: searching computer help using screenshots and keywords
Proceedings of the 20th international conference on World wide web
Knowing funny: genre perception and categorization in social video sharing
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
I will do it, but i don't like it: user reactions to preference-inconsistent recommendations
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Affective computational priming and creativity
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Social media ownership: using twitter as a window onto current attitudes and beliefs
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Cooks or cobblers?: crowd creativity through combination
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Field trial of Tiramisu: crowd-sourcing bus arrival times to spur co-design
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Playable data: characterizing the design space of game-y infographics
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Utility of human-computer interactions: toward a science of preference measurement
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing and human computation: systems, studies and platforms
CHI '11 Extended Abstracts on Human Factors in Computing Systems
Mobile phones and information capture in the workplace
CHI '11 Extended Abstracts on Human Factors in Computing Systems
Shepherding the crowd: managing and providing feedback to crowd workers
CHI '11 Extended Abstracts on Human Factors in Computing Systems
CrowdForge: crowdsourcing complex work
CHI '11 Extended Abstracts on Human Factors in Computing Systems
ViewSer: a tool for large-scale remote studies of web search result examination
CHI '11 Extended Abstracts on Human Factors in Computing Systems
The ownership and reuse of visual media
Proceedings of the 11th annual international ACM/IEEE joint conference on Digital libraries
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Computers in Human Behavior
Crowdsourcing for book search evaluation: impact of hit design on comparative system ranking
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
ViewSer: enabling large-scale remote user studies of web search examination and interaction
Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Personifying programming tool feedback improves novice programmers' learning
Proceedings of the seventh international workshop on Computing education research
Feature selection in crowd creativity
FAC'11 Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems
From consumers to owners: using meta-design environments to motivate changes in energy consumption
IS-EUD'11 Proceedings of the Third international conference on End-user development
Defensive research within the mobile marketplace
Proceedings of the 2nd international workshop on Research in the large
Emerging theories and models of human computation systems: a brief survey
Proceedings of the 2nd international workshop on Ubiquitous crowdsouring
Investigating intelligibility for uncertain context-aware applications
Proceedings of the 13th international conference on Ubiquitous computing
Second international workshop on ubiquitous crowdsourcing: towards a platform for crowd computing
Proceedings of the 13th international conference on Ubiquitous computing
Redundancy and collaboration in wikibooks
INTERACT'11 Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part I
Information quality assessment of community generated content: A user study of Wikipedia
Journal of Information Science
Instrumenting the crowd: using implicit behavioral measures to predict task performance
Proceedings of the 24th annual ACM symposium on User interface software and technology
CrowdForge: crowdsourcing complex work
Proceedings of the 24th annual ACM symposium on User interface software and technology
The design of human-powered access technology
The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility
Worker types and personality traits in crowdsourcing relevance labels
Proceedings of the 20th ACM international conference on Information and knowledge management
Collaborative information seeking by the numbers
Proceedings of the 3rd international workshop on Collaborative information retrieval
Learning by foraging: The impact of individual knowledge and social tags on web navigation processes
Computers in Human Behavior
Combining concept maps to catalyze creativity
C&C '11 Proceedings of the 8th ACM conference on Creativity and cognition
Crowd creativity through combination
C&C '11 Proceedings of the 8th ACM conference on Creativity and cognition
Crowdsourcing visual detectors for video search
MM '11 Proceedings of the 19th ACM international conference on Multimedia
Random partial paired comparison for subjective video quality assessment via hodgerank
MM '11 Proceedings of the 19th ACM international conference on Multimedia
Taking advantage of collective knowledge in emergency response systems
Journal of Network and Computer Applications
Proceedings of the Seventh Symposium on Usable Privacy and Security
"I regretted the minute I pressed share": a qualitative study of regrets on Facebook
Proceedings of the Seventh Symposium on Usable Privacy and Security
Do Mechanical Turks dream of square pie charts?
Proceedings of the 3rd BELIV'10 Workshop: BEyond time and errors: novel evaLuation methods for Information Visualization
Will you be my friend?: responses to friendship requests from strangers
Proceedings of the 2012 iConference
Supporting reflective public thought with considerit
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
Shepherding the crowd yields better work
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
CrowdWeaver: visually managing complex crowd work
Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
Consistency in physical and on-screen action improves perceptions of telepresence robots
HRI '12 Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction
Shreddr: pipelined paper digitization for low-resource organizations
Proceedings of the 2nd ACM Symposium on Computing for Development
Personality and persuasive technology: an exploratory study on health-promoting mobile applications
PERSUASIVE'10 Proceedings of the 5th international conference on Persuasive Technology
Pushing the boundaries of crowd-enabled databases with query-driven schema expansion
Proceedings of the VLDB Endowment
It's all about the benjamins: an empirical study on incentivizing users to ignore security advice
FC'11 Proceedings of the 15th international conference on Financial Cryptography and Data Security
Serf and turf: crowdturfing for fun and profit
Proceedings of the 21st international conference on World Wide Web
Collective intelligence as a source for machine learning self-supervision
Proceedings of the 4th International Workshop on Web Intelligence & Communities
Strategies for crowdsourcing social data analysis
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
On saliency, affect and focused attention
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Designing effective gaze mechanisms for virtual agents
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Unlocking the expressivity of point lights
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Socially computed scripts to support social problem solving skills
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
To switch or not to switch: understanding social influence in online choices
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Comparing averages in time series data
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CommunitySourcing: engaging local crowds to perform expert work via physical kiosks
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
LemonAid: selection-based crowdsourced contextual help for web applications
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
So who won?: dynamic max discovery with the crowd
SIGMOD '12 Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
An empirical test of the evidential reasoning approach's synthesis axioms
Expert Systems with Applications: An International Journal
A crowdsourcing quality control model for tasks distributed in parallel
CHI '12 Extended Abstracts on Human Factors in Computing Systems
On the institutional archiving of social media
Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries
Categorization of computing education resources with utilization of crowdsourcing
Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries
Generating ground truth for music mood classification using mechanical turk
Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries
Crowdsourcing semantic data management: challenges and opportunities
Proceedings of the 2nd International Conference on Web Intelligence, Mining and Semantics
Automatic identification of personal insults on social news sites
Journal of the American Society for Information Science and Technology
Graphical passwords: Learning from the first twelve years
ACM Computing Surveys (CSUR)
Correct horse battery staple: exploring the usability of system-assigned passphrases
Proceedings of the Eighth Symposium on Usable Privacy and Security
Effective temporal graph layout: a comparative study of animation versus static display methods
Information Visualization
CDAS: a crowdsourcing data analytics system
Proceedings of the VLDB Endowment
A human study of patch maintainability
Proceedings of the 2012 International Symposium on Software Testing and Analysis
Proceedings of the 34th International Conference on Software Engineering
On the naturalness of software
Proceedings of the 34th International Conference on Software Engineering
Learning from crowds in the presence of schools of thought
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Incentives for truthful reporting in crowdsourcing
Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems - Volume 3
How does your password measure up? the effect of strength meters on password creation
Security'12 Proceedings of the 21st USENIX conference on Security symposium
Crowdsourced web engineering and design
ICWE'12 Proceedings of the 12th international conference on Web Engineering
Crowdsourced web site evaluation with crowdstudy
ICWE'12 Proceedings of the 12th international conference on Web Engineering
Using crowdsourcing for TREC relevance assessment
Information Processing and Management: an International Journal
Navigating by index and guided tour for fact finding
Proceedings of the 30th ACM international conference on Design of communication
CrowdScape: interactively visualizing user behavior and output
Proceedings of the 25th annual ACM symposium on User interface software and technology
A personal touch: recognizing users based on touch screen behavior
Proceedings of the Third International Workshop on Sensing Applications on Mobile Phones
Workflow transparency in a microtask marketplace
Proceedings of the 17th ACM international conference on Supporting group work
Physical and spiritual proximity: linking Car2X communication with online social networks
Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
Tagging tagged images: on the impact of existing annotations on image tagging
Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia
Pushing the limits of mechanical turk: qualifying the crowd for video geo-location
Proceedings of the ACM multimedia 2012 workshop on Crowdsourcing for multimedia
Combining human and computation intelligence: the case of data interlinking tools
International Journal of Metadata, Semantics and Ontologies
Proceedings of the 21st ACM international conference on Information and knowledge management
Bannerbattle: introducing crowd experience to interaction design
Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design
A head-eye coordination model for animating gaze shifts of virtual characters
Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction
IVA'12 Proceedings of the 12th international conference on Intelligent Virtual Agents
A live comparison of methods for personalized article recommendation at forbes.com
ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Online real-time presentation of virtual experiences forexternal viewers
Proceedings of the 18th ACM symposium on Virtual reality software and technology
On automated image choice for secure and usable graphical passwords
Proceedings of the 28th Annual Computer Security Applications Conference
CrowdMap: crowdsourcing ontology alignment with microtasks
ISWC'12 Proceedings of the 11th international conference on The Semantic Web - Volume Part I
Swayed by friends or by the crowd?
SocInfo'12 Proceedings of the 4th international conference on Social Informatics
Your process is showing: controversy management and perceived quality in wikipedia
Proceedings of the 2013 conference on Computer supported cooperative work
Crowd vs. crowd: large-scale cooperative design through open team competition
Proceedings of the 2013 conference on Computer supported cooperative work
Co-worker transparency in a microtask marketplace
Proceedings of the 2013 conference on Computer supported cooperative work
Proceedings of the 2013 conference on Computer supported cooperative work
How to filter out random clickers in a crowdsourcing-based study?
Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization
Emergent semantics from game-induced folksonomies
Proceedings of the First International Workshop on Crowdsourcing and Data Mining
Wally: crowd powered image matching on tablets
Proceedings of the First International Workshop on Crowdsourcing and Data Mining
Learning to predict from textual data
Journal of Artificial Intelligence Research
Perspectives on crowdsourcing annotations for natural language processing
Language Resources and Evaluation
An internet-scale idea generation system
ACM Transactions on Interactive Intelligent Systems (TiiS) - Special section on internet-scale human problem solving and regular papers
Lessons Learned from Large-Scale User Studies: Using Android Market as a Source of Data
International Journal of Mobile Human Computer Interaction
Helping users with information disclosure decisions: potential for adaptation
Proceedings of the 2013 international conference on Intelligent user interfaces
An introduction to crowdsourcing for language and multimedia technology research
PROMISE'12 Proceedings of the 2012 international conference on Information Retrieval Meets Information Visualization
Influence of timeline and named-entity components on user engagement
ECIR'13 Proceedings of the 35th European conference on Advances in Information Retrieval
Crowdsourcing for information retrieval: introduction to the special issue
Information Retrieval
Increasing cheat robustness of crowdsourcing tasks
Information Retrieval
An analysis of human factors and label accuracy in crowdsourcing relevance judgments
Information Retrieval
Identifying top news using crowdsourcing
Information Retrieval
Implementing crowdsourcing-based relevance experimentation: an industrial perspective
Information Retrieval
Tagging human activities in video by crowdsourcing
Proceedings of the 3rd ACM conference on International conference on multimedia retrieval
Are user-contributed reviews community property?: exploring the beliefs and practices of reviewers
Proceedings of the 5th Annual ACM Web Science Conference
Proceedings of the 5th Annual ACM Web Science Conference
Experiences surveying the crowd: reflections on methods, participation, and reliability
Proceedings of the 5th Annual ACM Web Science Conference
An online cost sensitive decision-making method in crowdsourcing systems
Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data
Priming creativity through improvisation on an adaptive musical instrument
Proceedings of the 9th ACM Conference on Creativity & Cognition
Maater: crowdsourcing to improve online journalism
CHI '13 Extended Abstracts on Human Factors in Computing Systems
Form digitization in BPO: from outsourcing to crowdsourcing?
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Crowdsourcing performance evaluations of user interfaces
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A pilot study of using crowds in the classroom
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
CrowdStudy: general toolkit for crowdsourced evaluation of web interfaces
Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems
Analyzing users' narratives to understand experience with interactive products
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Preference-based location sharing: are more privacy options really better?
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Quantity estimation in visualizations of tagged text
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Juggling the Jigsaw: towards automated problem inference from network trouble tickets
nsdi'13 Proceedings of the 10th USENIX conference on Networked Systems Design and Implementation
Paraphrase acquisition via crowdsourcing and machine learning
ACM Transactions on Intelligent Systems and Technology (TIST) - Special Sections on Paraphrasing; Intelligent Systems for Socially Aware Computing; Social Computing, Behavioral-Cultural Modeling, and Prediction
News vertical search: when and what to display to users
Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval
An agent design for repeated negotiation and information revelation with people: Extended Abstract
Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems
Building test suites in social coding sites by leveraging drive-by commits
Proceedings of the 2013 International Conference on Software Engineering
Proceedings of the 22nd international conference on World Wide Web
Saving, reusing, and remixing web video: using attitudes and practices to reveal social norms
Proceedings of the 22nd international conference on World Wide Web
Pricing mechanisms for crowdsourcing markets
Proceedings of the 22nd international conference on World Wide Web
Managing distractions in complex settings
Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing
Games with a Purpose or Mechanised Labour?: A Comparative Study
Proceedings of the 13th International Conference on Knowledge Management and Knowledge Technologies
From sensing to controlling: the state of the art in ubiquitous crowdsourcing
International Journal of Communication Networks and Distributed Systems
Making Decisions about Privacy: Information Disclosure in Context-Aware Recommender Systems
ACM Transactions on Interactive Intelligent Systems (TiiS)
Rating Bias and Preference Acquisition
ACM Transactions on Interactive Intelligent Systems (TiiS)
Predicting time-sensitive user locations from social media
Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining
Your attention please: designing security-decision UIs to make genuine risks harder to ignore
Proceedings of the Ninth Symposium on Usable Privacy and Security
Proceedings of the Ninth Symposium on Usable Privacy and Security
Memory retrieval and graphical passwords
Proceedings of the Ninth Symposium on Usable Privacy and Security
CrowdLearner: rapidly creating mobile recognizers using crowdsourcing
Proceedings of the 26th annual ACM symposium on User interface software and technology
Robust evaluation for quality of experience in crowdsourcing
Proceedings of the 21st ACM international conference on Multimedia
Human vs machine: establishing a human baseline for multimodal location estimation
Proceedings of the 21st ACM international conference on Multimedia
1000 songs for emotional analysis of music
Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia
Proceedings of the 2nd ACM international workshop on Crowdsourcing for multimedia
Combining human and computation intelligence: the case of data interlinking tools
International Journal of Metadata, Semantics and Ontologies
The post anachronism: the temporal dimension of facebook privacy
Proceedings of the 12th ACM workshop on Workshop on privacy in the electronic society
Proceedings of the 2013 ACM workshop on Digital identity management
Feasibility of identifying eating moments from first-person images leveraging human computation
Proceedings of the 4th International SenseCam & Pervasive Imaging Conference
Evaluating a web-based tool for crowdsourced navigation stress tests
DUXU'13 Proceedings of the Second international conference on Design, User Experience, and Usability: web, mobile, and product design - Volume Part IV
Voyant: generating structured feedback on visual designs using a crowd of non-experts
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
The motivations and experiences of the on-demand mobile workforce
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Understanding the role of community in crowdfunding work
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Reviewing versus doing: learning and performance in crowd assessment
Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Mobility and social networking: a data management perspective
Proceedings of the VLDB Endowment
Accurate integration of crowdsourced labels using workers' self-reported confidence scores
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Complementing text entry evaluations with a composition task
ACM Transactions on Computer-Human Interaction (TOCHI)
The notion of diversity in graphical entity summarisation on semantic knowledge graphs
Journal of Intelligent Information Systems
Sharing Knowledge and Expertise: The CSCW View of Knowledge Management
Computer Supported Cooperative Work
POETIC: Interactive solutions to alleviate the reversal error in student-professor type problems
International Journal of Human-Computer Studies
Editorial: Special Issue on Advances in Computer Supported Collaboration: Systems and Technologies
Future Generation Computer Systems
Expert Systems with Applications: An International Journal
STFU NOOB!: predicting crowdsourced decisions on toxic behavior in online games
Proceedings of the 23rd international conference on World wide web
Newsvendor pull-to-center reconsidered
Decision Support Systems
Crowdsourced Knowledge Acquisition: Towards Hybrid-Genre Workflows
International Journal on Semantic Web & Information Systems
Evaluation in Music Information Retrieval
Journal of Intelligent Information Systems
Hi-index | 0.01 |
User studies are important for many aspects of the design process and involve techniques ranging from informal surveys to rigorous laboratory studies. However, the costs involved in engaging users often requires practitioners to trade off between sample size, time requirements, and monetary costs. Micro-task markets, such as Amazon's Mechanical Turk, offer a potential paradigm for engaging a large number of users for low time and monetary costs. Here we investigate the utility of a micro-task market for collecting user measurements, and discuss design considerations for developing remote micro user evaluation tasks. Although micro-task markets have great potential for rapidly collecting user measurements at low costs, we found that special care is needed in formulating tasks in order to harness the capabilities of the approach.