Thoughts on Hesse, Digital Art and Visual Music
This essay was first published in the 2004 SIGGRAPH Electronic Art and Animation Catalog.
Thoughts on Hesse, Digital Art and Visual Music
Excerpts from the Variations artist statements (1999-2003)
by Bruce Wands
This essay describes the influence of Herman Hesse’s ideas on my creative work and how I create interactive music sculptures and visual music. In his book, Magister Ludi, Hesse describes a game in which art and music blend together in a game that allows for the transformation of creative content into various forms of media. Varations is an ongoing exploration of interactive sculpture and visual music that began in 1999 and still continues. The following describes my thought process for early versions of this work, as well as for Variations 03, an interactive music installation which was exhibited as part of the SIGGRAPH 2003 Art Gallery. My approach in creating this work was to develop a three dimensional sound matrix that was changeable by viewers as they interacted with the sculpture.
The following quotes will give some insight into why I chose to create art the way I do:
“I very often reread books that have made an impression on me. Several years ago, while on my way to the SIGGRAPH 94 computer graphics conference, I brought along a copy of Magister Ludi (also published under the title The Glass Bead Game) by Hermann Hesse to read on the plane. The following quotes stood out from the rest of the text and prompted me to stop and re-evaluate my creative work and the field of digital art in general.” (1)
“In the formal game, the player sought to compose out of the objective content of every game, out of the mathematical, linguistic, musical and other elements, as dense, coherent, and formally perfect a unity as possible…”
“(O)ne can be a musician or Glass Bead Game player and at the same time wholly devoted to rule and order. The kind of person we want to develop, the kind of person we aim to become, would at any time be able to exchange his discipline or art for any other.” – Hermann Hesse, Magister Ludi (2)
“Having a background as a musician, producer, photographer and visual artist, I had long thought of myself as a Glass Bead Game player. As digital art continues to become the new art form of the next millennium, I am now even more convinced that Hesse’s remarks were indeed prophetic. (After all, this book did win him the Nobel Prize for Literature in 1943.)
When I reread Hesse’s novel, I was in the process of curating the Second Annual New York Digital Salon, and it struck me that digital art has a Glass Bead Game component to it. In the intervening years, there have been considerable advances in digital imaging, animation software, digital audio/video and the global explosion of the Internet. Access to this technology has become almost ubiquitous and artists are now frequently starting to branch out into media that were not their original primary focus: painters are creating Web sites with interactive and textual components, while photographers are experimenting with video and 3D animation. The internet is also acting as a means of leveling the playing field for artists. It is an inclusive medium that allows text, images, sound and video to be contained within a single artist’s Web site.
This branching out has positive and negative connotations for art. It is allowing artists more creative freedom and giving them access to new tools, but it runs the risk of enabling individuals to create work in media in which they do not have a well-developed vocabulary or aesthetic foundation. The question then arises as to whether the work created by these artists is a form of the Glass Bead Game or simply a byproduct of the curiosity of the creative imagination.
Despite the current sophistication in hardware and software, the fact remains that the fundamental element of digital art is data. All images, words, sounds, video and text are ultimately reduced to a string of ones and zeros, stored in a digital file on the computer. On the transistor level, the circuit is either closed or open, in a sense coupled or uncoupled. Depending on the type of media stored, when the data is read by the computer it is converted by ASCII code to make it text, image file formats to make it visible, or digital audio formats to make it aural.
Hesse’s reference to the “formal game” can also be interpreted as referring to the computer, which makes visual, aural and textual data as “dense, coherent, and formally perfect a unity as possible,” not in the symbolic or intellectual sense, but in a practical way – to allow in to be stored and transferred from machine to machine, media to media. One of the benefits of digitizing creative work is that it provides artists a way to access and manipulate this data and to reinterpret it within the context of their individual creative work. (1)
Many artists and composers have been fascinated by the relationship between music and art. Artists have made visual interpretations of music and musicians have employed color, images and sound into their performances. Music is very efficacious at evoking moods and often stirs memories, resulting in various thoughts and often suggesting colors. When approaching the relationship between music and art, we can look at several issues: tempo, structure, composition, mathematical elements and creative interpretation. My approach to creating visual music is to experiment with these various components as they relate to the specific musical passage.
Tempo can be looked at as the guiding force behind the amount of detail or the density of the image. A slow tempo would suggest large shapes and very deliberate graphic elements. A fast tempo would imply the opposite, a fairly dense image with many components. These relate to how the eye looks at an image and how the ear interprets the sound. Fast tempos require a bit more concentration because there are many more notes in the music. Visually, the eye would be moving slowly or quickly across the image.
Structure relates to the compositional forms used by the composer. Popular music is typically divided into verse, chorus and bridge. Symphonies have several sections to them. The musical experience is by nature a temporal one. There is a beginning, middle and end. Taking this further, there may be repeated passages and variations on particular themes. When applying these principles to a visual interpretation of the music, we now can look at repetition of form and characteristics of the music. For example ascending and descending musical passages suggest lines or shapes that are rising and falling. A musical piece that repeats the same phrase throughout the piece would suggest an image that has a repetitive motif in it.
Composition in this context relates to the style of the music and the instrumentation used. In other words, how the composer chooses the tempo and structure and then weaves it into the particular musical composition that they are writing. This can take many forms depending on the cultural background of the composer and their approach to their work. Classical music is very formal and has very distinct structures. Jazz is based on improvisation and the music composition provides a jumping off point for the musicians to read their own interpretation into the music. Taking these last two ideas into a visual interpretation, classical music might suggest a more precise type of image, whereas jazz may suggest a more abstract approach to the image.
While composition is a subjective process, the mathematical interpretation of music is a very analytical one. While leaving the door open to the musical interpretation of the work, the mathematical elements are easily quantified. This has been made easier by the introduction of MIDI (musical instrument digital interface). MIDI was developed in the mid-1980s by synthesizer manufacturers to allow different instruments to talk to each other. MIDI looks at music through several parameters: note on, note off, pitch, velocity and after touch. This is radically different from the way music is stored on tape, CD’s or as a digital audio file in the computer. CD and digital audio files are stored in formats that relate to frequency and amplitude. For example, the standard CD format is 16 bit and 44.1 khz. 16 bit means that the resolution of the samples is high and that there are 44,100 samples of the audio taken per second. A digital file of this type gives the listener an exact reproduction of the sound. However, digital audio files do not give much to the artist. What one sees on the screen is a pattern that varies mainly with the amplitude of the music. There are several visual music software programs that give a visual depiction of the music. Taking this one step further, there are now software programs that allow one to “play” images in real time on the keyboard, along with the music.
Getting back to the mathematical analysis of a musical composition, MIDI has now made this process much easier. On a basic level, there are a definable number of notes in a musical piece. Their pitch and duration are quantified, along with the velocity and aftertouch, which are how hard the note is struck and how quickly or slowly the note is ended. By gathering this data, the visual artist can now deconstruct the music into a group of numbers that represent the music. Conversely, these numbers can now be manipulated by the artist into an image that carries the fundamental units of the music within it. For example, low notes can be large spheres and high notes can be smaller spheres. Or, low notes can be darker colors, while higher notes can be lighter colors.
The final element to be considered when discussing visual music is creative interpretation. Although much of the above has related to the visual characteristics of the music and the underlying mathematics of the music, creative interpretation is the point where the artist must take liberties with the data to create a work of art that has aesthetic value. A simple visualization of the mathematical or structural elements of a musical piece is not art, as is a rote performance of the music as it is written not “real” music. It is the musician who makes the sheet music turn into the musical experience. It is the artist who must take all of these elements of the music and turn them into art. Visual music can take the many forms that art has available to it. Visual music has been interpreted into sculptures, images and animation.
Continuing from the Leonardo essay, “I would like to refer to my own creative work as another example of the deliberate use of the synesthetic approach of coupling and uncoupling data and sensory information with meaning as a tool for inspiration.” I am currently working on a series of interactive musical sculptures, and the following describes my creative thought process for Variations 03 , an interactive music installation which was exhibited as part of the SIGGRAPH 2003 Art Gallery and for visual music in general. My approach in creating this work was to develop a three dimensional sound matrix that was changeable by viewers as they interact with the sculpture.
The first step in the process was to look at the musical phrase in terms of its mathematical content. I counted the number of musical measures and the number and pitch of the notes to develop for myself a mathematical representation of the musical phrase.
My next step will be to translate the score into MIDI (Musical Instrument Digital Interface) data through a performance of the work to allow myself to look more closely at the musical interpretation aspects of the piece. MIDI allows the recording of musical parameters like pitch, duration, velocity and aftertouch. These last two parameters define how hard a note is struck and how fast or slowly the note is released. Having the data in MIDI format now allows me to digitally control the performance to produced a large number of variations on this theme. For example, by speeding up and slowing down the tempo and by using a wide range of instrumental voices to play the musical phrase, I can hear it in many different ways. By creating these variations, I can explore the underlying rhythms and structure of the music from an aural point of view.
Once this has been accomplished, I will look at the graphic elements of the music. Musical notation is by nature a visual picture of sound. One of the shortcomings of musical notation is that it evolved in a period of acoustic instrumentation. Although open to interpretation by various musicians, the palette of instruments available at the time was limited. Several new notation systems have been developed over the past several years for electronic and digital music, although no standard has emerged; these new systems are more often that not a way for individual composers to visually document their music. In my case, the original score will be translated into a 3D software package based on the number, pitch and duration of the notes. I will use the numeric data as supplied by the MIDI file. In addition, I usually sketch ideas while listening to the various interpretations I have recorded of the musical phrase.
Out of this process starts to emerge a visual picture of the music and a design for the sculpture. The music has evolved from the written score, to a MIDI performance with numerous interpretations to color sketches made while listening to the music; from there it develops into a three-dimensional database. This database allows me to visually play with the data, the same way that I played with it musically. I can apply different geometrical shapes and color to the data, and can view the data from a variety of viewpoints…. Notes from the score were translated into MIDI data, and then entered into Alias PowerAnimator software. … I will use this same process with the data generated from the music, resulting in a design for a sculpture. (1)
Digital art allows artists more creative freedom. Fro example, music data can be re-interpreted as an image, sculpture, and vice versa. My goal in creating the piece, Variations 703, was to open the imagination of the viewers. Rather than hearing a single recorded interpretation of the music, they now hear several instrumental interpretations. They also see an artistic interpretation of the music in abstract imagery, and/or interact with a sculptural interface. The Glass Bead Game invented by Hermann Hesse allows for the symbolic transformation of the original creative inspiration of a work of art, music or literary work into the various art forms and media that artists create. One of the purposes of art is to change the viewer’s perception. It is hoped that after the viewer experiences Variations 703, they will think differently about music, sculpture and art.
1. Bruce Wands, “Digital Salon Chair’s Statement”, LEONARDO, Vol. 32, Number 5, pp. 349-351, MIT Press, 1999
2. Hermann Hesse, Magister Ludi (the Glass Bead Game), trans. Richard and Clara Winston (New York: Bantam, 1986 (1969)), pp. 68, 178.