MelodicBrush: a cross-modal link between ancient and digital art forms

Authors:
Michael Xuelin Huang;Will Tang;Kenneth W.K. Lo;C.K. Lau;Grace Ngai;Stephen Chan
Affiliations:
The Hong Kong Polytechnic University, Kowloon, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong;The Hong Kong Polytechnic University, Kowloon, Hong Kong
Venue:
CHI '12 Extended Abstracts on Human Factors in Computing Systems
Year:
2012

Citing 8
Cited 0

A flexible system for creating music while interacting with the computer

Proceedings of the 13th annual ACM international conference on Multimedia
The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces

Proceedings of the 1st international conference on Tangible and embedded interaction
DrawSound: a drawing instrument for sound performance

Proceedings of the 2nd international conference on Tangible and embedded interaction
MySong: automatic accompaniment generation for vocal melodies

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Data-driven exploration of musical chord sequences

Proceedings of the 14th international conference on Intelligent user interfaces
FluidPaint: an interactive digital painting system using real wet brushes

Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces
Using a depth camera as a touch sensor

ACM International Conference on Interactive Tabletops and Surfaces
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera

Proceedings of the 24th annual ACM symposium on User interface software and technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

MelodicBrush is a novel cross-modal musical system that connects two ancient art forms: Chinese ink-brush calligraphy and Chinese music. Our system endows the process of calligraphy writing with a novel auditory representation in a natural and intuitive manner to create a novel artistic experience. The writing effect is simulated as though the user were writing on an infinitely large piece of paper viewed through a viewport. The real-time musical generation effects are motivated by principles of metaphoric congruence and statistical music modeling.