The purge threat: scientists' thoughts on peta-scale usability

• Authors: Alexandra Holloway
• Venue: Proceedings of the sixth workshop on Parallel Data Storage
• Year: 2011

In high-performance scientific computing, users output millions of files per project or simulation, resulting in petabytes of information. Little is known about how users make sense of it all, and what the major usability issues are in interacting with a file system at scale. We conducted interviews with scientists at national laboratories to identify common practices and issues with current peta-scale file system usage. The major usability problem encountered in the interviews was the purge threat, triggered when the parallel file system reaches capacity, and warning users about impending data loss. We show that the threat is not communicated to the users of the system in a meaningful way. We present three methods scientists used to address the purge threat--analysis, automation, and subversion--and discuss how subversion of the purging system is a clear indication of its lack of utility and indicative of its cognitive complexity. We define reactionary and cautionary archiving and draw a parallel between archiving methods and data production paradigms. Finally, we propose two non-hierarchical file and directory representation models to address the purge threat.