User interface evaluation in the real world: a comparison of four techniques
CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Organizational obstacles to interface design and development: two participant-observer studies
ACM Transactions on Computer-Human Interaction (TOCHI)
Usability inspection methods
Methodology matters: doing research in the behavioral and social sciences
Human-computer interaction
Contextual design: defining customer-centered systems
Contextual design: defining customer-centered systems
Empirical evaluation of information visualizations: an introduction
International Journal of Human-Computer Studies - Empirical evaluation of information visualizations
Beyond Interaction Design: Beyond Human-Computer Interaction
Beyond Interaction Design: Beyond Human-Computer Interaction
Software technology in an automotive company: major challenges
Proceedings of the 25th International Conference on Software Engineering
Benefits of Information Visualization Systems for Administrative Data Analysts
IV '03 Proceedings of the Seventh International Conference on Information Visualization
Human Factors in Visualization Research
IEEE Transactions on Visualization and Computer Graphics
The challenge of information visualization evaluation
Proceedings of the working conference on Advanced visual interfaces
The state of user-centered design practice
Communications of the ACM - The disappearing computer
Automotive System Design - Challenges and Potential
Proceedings of the conference on Design, Automation and Test in Europe - Volume 1
An Insight-Based Methodology for Evaluating Bioinformatics Visualizations
IEEE Transactions on Visualization and Computer Graphics
Evaluating Visualizations: Do Expert Reviews Work?
IEEE Computer Graphics and Applications
Challenges in automotive software engineering
Proceedings of the 28th international conference on Software engineering
An explorative analysis of user evaluation studies in information visualisation
Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization
Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization
An Insight-Based Longitudinal Study of Visual Analytics
IEEE Transactions on Visualization and Computer Graphics
Designing Interactions
Human-Computer Interaction (3rd Edition)
Human-Computer Interaction (3rd Edition)
Software Engineering for Automotive Systems: A Roadmap
FOSE '07 2007 Future of Software Engineering
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
LiveRAC: interactive visual exploration of system management time-series data
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Productivity as a metric for visual analytics: reflections on e-discovery
Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization
Increasing the utility of quantitative empirical studies for meta-analysis
Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization
Grounded evaluation of information visualizations
Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization
Qualitative analysis of visualization: a building design field study
Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization
Evaluating Information Visualizations
Information Visualization
A Dual-View Visualization of In-Car Communication Processes
IV '08 Proceedings of the 2008 12th International Conference Information Visualisation
MUST: a method for participatory design
Human-Computer Interaction
Obstacles to usability evaluation in practice: a survey of software development organizations
Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges
Driving user centered design into IT organizations: is it possible?
CHI '09 Extended Abstracts on Human Factors in Computing Systems
Towards the Big Picture: Enriching 3D Models with Information Visualisation and Vice Versa
SG '09 Proceedings of the 10th International Symposium on Smart Graphics
User-Centered Development of a Visual Exploration System for In-Car Communication
SG '09 Proceedings of the 10th International Symposium on Smart Graphics
MostVis: An Interactive Visualization Supporting Automotive Engineers in MOST Catalog Exploration
IV '09 Proceedings of the 2009 13th International Conference Information Visualisation
A Nested Model for Visualization Design and Validation
IEEE Transactions on Visualization and Computer Graphics
Evaluating information visualization in large companies: challenges, experiences and recommendations
Proceedings of the 3rd BELIV'10 Workshop: BEyond time and errors: novel evaLuation methods for Information Visualization
Comparative Visual Analysis of 2D Function Ensembles
Computer Graphics Forum
The four-level nested model revisited: blocks and guidelines
Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization
Hi-index | 0.00 |
We examine the implications of evaluating data analysis processes and information visualization tools in a Large company setting. While several researchers have addressed the difficulties of evaluating information visualizations with regards to changing data, tasks, and visual encodings, considerably Less work has been published on the difficulties of evaluation within specific work contexts. We specifically focus on the challenges, which arise in the context of large companies with several thousand employees. Based on your own experience from a 3.5-year collaboration within a Large automotive company, we first present a collection of nine information visualization evaluation challenges. We then discuss these challenges by means of two concrete visualization case studies from our own work. We finally derive a set of 16 recommendations for planning and conducting evaluations in Large company settings. The set of challenges and recommendations and the discussion of our experience are meant to provide practical guidance to other researchers and practitioners, who plan to study information visualization in large company settings.