HueGaze: The Art of Scientific Mapping : Cross-modal Sonification of Images

Sammanfattning

This master’s thesis explores the relationship between vision and hearing by investigating how cross-modal correspondences can serve as a framework for musical composition through sonification. Central to this project is the question of how the mapping process is shaped by the interplay between artistic and technical decision-making within a sonification framework. An algorithm was designed and evaluated, titled HueGaze, which translates visual attributes from digital images into musical parameters grounded in human perception. Developed in Python and SuperCollider, the algorithm uses the CIELAB colour space to extract perceptual data such as colours, lightness and entropy from an image. These values are mapped to acoustic parameters such as pitch, amplitude, and timbre, inspired by established cross-modal correspondences. Research through Design was applied as the overarching methodology, enabling theory, technical implementation, and artistic reflection to interact iteratively throughout the process. The results demonstrated that the algorithm transformed static images into dynamic musical compositions, realizing the project's concept of the image as a visual score. In doing so, a central tension surfaced between the scientific objectivity traditionally required by sonification and the artistic judgment inherent in compositional decision-making. More broadly, this reflects a methodological challenge within cross-modal research, where neither strict empirical nor purely artistic approaches alone can adequately address questions rooted in shared subjective experience. The primary contribution of this work is to frame this tension, not as a limitation to overcome, but as the working ground in which cross-modal design takes place. Through what is articulated as the aesthetic filter, this project presents a set of design principles for navigating the interplay between technical and artistic judgment. By applying these principles, the image is treated less as raw data, and more as a musical score to be read.