ButterflyNet: a mobile capture and access system for field biology research

Authors:
Ron Yeh;Chunyuan Liao;Scott Klemmer;François Guimbretière;Brian Lee;Boyko Kakaradov;Jeannie Stamberger;Andreas Paepcke
Affiliations:
Stanford University;University of Maryland;Stanford University;University of Maryland;Stanford University;Stanford University;Stanford University;Stanford University
Venue:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Year:
2006

Citing 25
Cited 97

Bridging the paper and electronic worlds: the paper user interface

CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
FILOCHAT: handwritten notes provide access to recorded conversations

CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
“I'll get that off the audio”: a case study of salvaging multimedia meeting records

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Reinventing the familiar: exploring an augmented reality design space for air traffic control

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Palette: a paper interface for giving presentations

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Personalizing the capture of public experiences

Proceedings of the 12th annual ACM symposium on User interface software and technology
Linking and messaging from real paper in the Paper PDA

Proceedings of the 12th annual ACM symposium on User interface software and technology
Past, present, and future of user interface software tools

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction in the new millennium, Part 1
Charting past, present, and future research in ubiquitous computing

ACM Transactions on Computer-Human Interaction (TOCHI) - Special issue on human-computer interaction in the new millennium, Part 1
Listen reader: an electronically augmented paper-based book

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The audio notebook: paper and pen interaction with structured speech

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Guided gesture support in the paper PDA

Proceedings of the 14th annual ACM symposium on User interface software and technology
The missing link: augmenting biology laboratory notebooks

Proceedings of the 15th annual ACM symposium on User interface software and technology
Labscape: A Smart Environment for the Cell Biology Laboratory

IEEE Pervasive Computing
Books with voices: paper transcripts as a physical interface to oral histories

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent Paper

EP '98/RIDT '98 Proceedings of the 7th International Conference on Electronic Publishing, Held Jointly with the 4th International Conference on Raster Imaging and Digital Typography: Electronic Publishing, Artistic Imaging, and Digital Typography
The Myth of the Paperless Office

The Myth of the Paperless Office
Tangible multimodal interfaces for safety-critical applications

Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Paper augmented digital documents

Proceedings of the 16th annual ACM symposium on User interface software and technology
Papier-Mache: toolkit support for tangible input

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Toolkit Design for Interactive Structured Graphics

IEEE Transactions on Software Engineering
Only touching the surface: creating affinities between digital content and paper

CSCW '04 Proceedings of the 2004 ACM conference on Computer supported cooperative work
Livenotes: a system for cooperative and augmented note-taking in lectures

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Using Mobile Phones for Secure, Distributed Document Processing in the Developing World

IEEE Pervasive Computing
PapierCraft: a command system for interactive paper

Proceedings of the 18th annual ACM symposium on User interface software and technology

Technology for design education: a case study

CHI '06 Extended Abstracts on Human Factors in Computing Systems
Physical-digital ensembles for mobile interaction

CHI '06 Extended Abstracts on Human Factors in Computing Systems
Design research for a context-aware capture system to support biology education

DIS '06 Proceedings of the 6th conference on Designing Interactive systems
Pen-top feedback for paper-based interfaces

UIST '06 Proceedings of the 19th annual ACM symposium on User interface software and technology
Collaborative multimodal photo annotation over digital paper

Proceedings of the 8th international conference on Multimodal interfaces
Designing a mobile user interface for automated species identification

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Mobile G-Portal supporting collaborative sharing and learning in geography fieldwork: an empirical study

Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries
Pen-based interaction

ACM SIGGRAPH 2007 courses
Data model and architecture of a paper-digital document management system

Proceedings of the 2007 ACM symposium on Document engineering
Papiercraft: A gesture-based command system for interactive paper

ACM Transactions on Computer-Human Interaction (TOCHI)
Premote: a user customizable remote control

CHI '08 Extended Abstracts on Human Factors in Computing Systems
Studying paper use to inform the design of personal and portable technology

CHI '08 Extended Abstracts on Human Factors in Computing Systems
Forty years of research in character and document recognition-an industrial perspective

Pattern Recognition
Bridging the gap between real printouts and digital whiteboard

AVI '08 Proceedings of the working conference on Advanced visual interfaces
Information scraps: How and why information eludes our personal information management tools

ACM Transactions on Information Systems (TOIS)
Interactionist AI and the promise of ubicomp, or, how to put your box in the world without putting the world in your box

UbiComp '08 Proceedings of the 10th international conference on Ubiquitous computing
Spyn: augmenting knitting to support storytelling and reflection

UbiComp '08 Proceedings of the 10th international conference on Ubiquitous computing
Iterative design and evaluation of an event architecture for pen-and-paper interfaces

Proceedings of the 21st annual ACM symposium on User interface software and technology
A high-performance dual-wizard infrastructure for designing speech, pen, and multimodal interfaces

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
HOTPAPER: multimedia interaction with paper using mobile phones

MM '08 Proceedings of the 16th ACM international conference on Multimedia
From individual to collaborative: the evolution of prism, a hybrid laboratory notebook

Proceedings of the 2008 ACM conference on Computer supported cooperative work
Espace de caractérisation du stylo numérique

Proceedings of the 20th International Conference of the Association Francophone d'Interaction Homme-Machine
Designing pen-and-paper user interfaces for interaction with documents

Proceedings of the 3rd International Conference on Tangible and Embedded Interaction
PenLight: combining a mobile projector and a digital pen for dynamic visual overlay

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Musink: composing music through augmented drawing

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Augmenting paper-based learning with mobile phones

Interacting with Computers
Tangible interactions in a digital age: Medium and graphic visualization in design journals

Artificial Intelligence for Engineering Design, Analysis and Manufacturing
Ubiquitous Computing for Capture and Access

Foundations and Trends in Human-Computer Interaction
Patterns of collaboration in design courses: team dynamics affect technology appropriation, artifact creation, and course performance

CSCL'07 Proceedings of the 8th iternational conference on Computer supported collaborative learning
Elicitation of User Requirements for Mobile Interaction with Visual and RFID Tags: A Prototype-Based Exploratory Study

HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
Understanding Activity Documentation Work in Remote Mobility Environments

HCD 09 Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International 2009
Mobile Technology for Children: Designing for Interaction and Learning

Mobile Technology for Children: Designing for Interaction and Learning
U-Note: classe augmentée et stylo numérique

Proceedings of the 21st International Conference on Association Francophone d'Interaction Homme-Machine
L'écriture augmentée: enregistrer des explorations interactives avec une feuille de données scientifiques

Proceedings of the 21st International Conference on Association Francophone d'Interaction Homme-Machine
Reflections on craft: probing the creative process of everyday knitters

Proceedings of the seventh ACM conference on Creativity and cognition
Sharing Mobile Multimedia Annotations to Support Inquiry-Based Learning Using MobiTOP

AMT '09 Proceedings of the 5th International Conference on Active Media Technology
Cartouche: conventions for tangibles bridging diverse interactive systems

Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction
Enhancing learning: a study of how mobile devices can facilitate sensemaking

Personal and Ubiquitous Computing
FrameWire: a tool for automatically extracting interaction logic from paper prototyping tests

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
NiCEBook: supporting natural note taking

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
G-nome surfer: a tabletop interface for collaborative exploration of genomic data

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Spyn: augmenting the creative and communicative potential of craft

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Pacer: fine-grained interactive paper via camera-touch hybrid gestures on a cell phone

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
MouseLight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Augmented paper: developing relationships between digital content and paper

The disappearing computer
Why it's worth the hassle: the value of in-situ studies when designing Ubicomp

UbiComp '07 Proceedings of the 9th international conference on Ubiquitous computing
PaperCP: exploring the integration of physical and digital affordances for active learning

INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction - Volume Part II
Mobile tagging and accessibility information sharing using a geospatial digital library

ICADL'07 Proceedings of the 10th international conference on Asian digital libraries: looking back 10 years and forging new frontiers
Understanding and supporting lightweight communication in hospital work

IEEE Transactions on Information Technology in Biomedicine
Letras: an architecture and framework for ubiquitous pen-and-paper interaction

Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems
Knotty gestures: subtle traces to support interactive use of paper

Proceedings of the International Conference on Advanced Visual Interfaces
Supporting self-expression for informal communication

Proceedings of the 12th ACM international conference adjunct papers on Ubiquitous computing - Adjunct
Supporting self-expression for informal communication

UIST '10 Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology
Enabling social interactions through real-time sketch-based communication

UIST '10 Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology
Introducing multimodal paper-digital interfaces for speech-language therapy

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility
Hybrid documents ease text corpus analysis for literary scholars

ACM International Conference on Interactive Tabletops and Surfaces
Supporting an integrated paper-digital workflow for observational research

Proceedings of the 16th international conference on Intelligent user interfaces
Eden: a professional multitouch tool for constructing virtual organic environments

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
GreenHat: exploring the natural environment through experts' perspectives

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Finders/keepers: a longitudinal study of people managing information scraps in a micro-note tool

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
W5: a meta-model for pen-and-paper interaction

Proceedings of the 3rd ACM SIGCHI symposium on Engineering interactive computing systems
An extensible digital ink segmentation and classification framework for natural notetaking

Proceedings of the 3rd ACM SIGCHI symposium on Engineering interactive computing systems
Using paper and pen to control home-IT: lessons learned by hands-on experience

Proceddings of the 9th international interactive conference on Interactive television
Field to file: a tool for activity documentation work in remote mobility environments

HCII'11 Proceedings of the 14th international conference on Human-computer interaction: towards mobile and intelligent interaction environments - Volume Part III
The hybrid shopping list: bridging the gap between physical and digital shopping lists

Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services
Ubiquitous sketching for social media

Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services
Write-N-Speak: Authoring Multimodal Digital-Paper Materials for Speech-Language Therapy

ACM Transactions on Accessible Computing (TACCESS)
U-note: capture the class and access it everywhere

INTERACT'11 Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part I
Etude exploratoire du stylo électronique pour le Contrôle Aérien

23rd French Speaking Conference on Human-Computer Interaction
Feeding the digital parrot: capturing situational context in an augmented memory system

Proceedings of the 23rd Australian Computer-Human Interaction Conference
The eLabBench: an interactive tabletop system for the biology laboratory

Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces
Putting location-based services on the map

W2GIS'06 Proceedings of the 6th international conference on Web and Wireless Geographical Information Systems
Embedded capture and access: encouraging recording and reviewing of data in the caregiving domain

Personal and Ubiquitous Computing
A pen-based toolkit for authoring collaborative language activities

Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion
Beyond data sharing: artifact ecology of a collaborative nanophotonics research centre

Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
A study of incidental notetaking to inform digital pen and paper solutions

BCS '10 Proceedings of the 24th BCS Interaction Specialist Group Conference
TAP & PLAY: an end-user toolkit for authoring interactive pen and paper language activities

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Interactive paper substrates to support musical creation

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Toward a theory of interaction in mobile paper-digital ensembles

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Digital pen and paper practices in observational research

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The eLabBench in the wild: supporting exploration in a molecular biology lab

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
PartoPen: enhancing the partograph with digital pen technology

CHI '12 Extended Abstracts on Human Factors in Computing Systems
PaperUI

CBDAR'11 Proceedings of the 4th international conference on Camera-Based Document Analysis and Recognition
S-Notebook: augmenting mobile devices with interactive paper for data management

Proceedings of the International Working Conference on Advanced Visual Interfaces
Enabling live dialogic and collaborative learning between field and indoor contexts

BCS-HCI '11 Proceedings of the 25th BCS Conference on Human-Computer Interaction
The design, development, and deployment of a tabletop interface for collaborative exploration of genomic data

International Journal of Human-Computer Studies
Interpreting strokes on paper with a mobile assistant

Proceedings of the 25th annual ACM symposium on User interface software and technology
The sociality of fieldwork: designing for social science research practice and collaboration

Proceedings of the 17th ACM international conference on Supporting group work
A collaborative environment for engaging novices in scientific inquiry

Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces
ARENO: ambient reminder notes

Proceedings of the 24th Australian Computer-Human Interaction Conference
Audio-enhanced paper photos: encouraging social interaction at age 105

Proceedings of the 2013 conference on Computer supported cooperative work
From big data to insights: opportunities and challenges for TEI in genomics

Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction
Reifying social movement trajectories

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Annotating ecology: looking to biological fieldwork for mobile spatial annotation workflows

Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services
Aiding human discovery of handwriting recognition errors

Proceedings of the 15th ACM on International conference on multimodal interaction
Study of electronic lab notebook design and practices that emerged in a collaborative scientific environment

Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing
Supporting Scientific Collaboration: Methods, Tools and Concepts

Computer Supported Cooperative Work

Quantified Score

Hi-index	0.01

Visualization

Abstract

Through a study of field biology practices, we observed that biology fieldwork generates a wealth of heterogeneous information, requiring substantial labor to coordinate and distill. To manage this data, biologists leverage a diverse set of tools, organizing their effort in paper notebooks. These observations motivated ButterflyNet, a mobile capture and access system that integrates paper notes with digital photographs captured during field research. Through ButterflyNet, the activity of leafing through a notebook expands to browsing all associated digital photos. ButterflyNet also facilitates the transfer of captured content to spreadsheets, enabling biologists to share their work. A first-use study with 14 biologists found this system to offer rich data capture and transformation, in a manner felicitous with current practice.