Visual programming with analogical representations: Inspirations from a semiotic analysis of comics

  • Authors:

  • Mikael Kindborg;Kevin McGee

  • Affiliations:

  • Department of Computer and Information Science, Linköping University, SE-58183 Linköping, Sweden;Department of Computer and Information Science, Linköping University, SE-58183 Linköping, Sweden

  • Venue:
  • Journal of Visual Languages and Computing
  • Year:
  • 2007

  • Cognitive aspects of programming in pictures

    IEA/AIE'11 Proceedings of the 24th international conference on Industrial engineering and other applications of applied intelligent systems conference on Modern approaches in applied intelligence - Volume Part II

  • Intelligent aspects of AIDA programming

    IEA/AIE'12 Proceedings of the 25th international conference on Industrial Engineering and Other Applications of Applied Intelligent Systems: advanced research in applied artificial intelligence

Quantified Score

Hi-index 0.01

Visualization

Abstract

Analogical representations based on pictures of domain objects can be used in visual programming to provide a close mapping between the program and the resulting runtime display, which can make programming easier for children and other users. The use of graphical rewrite rules with before and after pictures is an example of this approach. Graphical rewrite rules have some similarities with comics strips, which also use picture sequences of graphical objects to describe dynamics in a static form. However, the visual language of comics is not used to its full potential in visual programming. We discuss how a semiotic analysis of comics can be used to address some of the limitations of graphical rewrite rules. We use a visual programming system we have designed to illustrate that comic strips can express more general computations and be more intuitive and flexible than traditional graphical rewrites. Our conclusion is that the visual language of comics has a strong potential for increasing the expressiveness and flexibility of visual programming with analogical representations of domain objects, while maintaining a direct mapping between the program representation and the runtime representation.