Computers get emotional

Here’s more on Iamus, the computer programme that writes music you might actually want to listen to. It is published in Nature this week, although there are more references in this version. The CD of Iamus’ music is released in mid-September (not the 1st, as the Nature piece says), and is simply called Iamus (Melomics Records). I’m looking forward to that.

________________________________________________________________

If a computer can produce something that moves us, would this take artificial intelligence beyond an important threshold? That’s one of the questions seemingly raised by an algorithm called Iamus, developed by Francisco Vico and colleagues at the University of Malaga, which composes music from scratch.

There’s nothing new about computers making music. They have been used by composers since the early days of computer technology in the 1960s, most notably by the experimental Greek composer Iannis Xenakis. Neither is there anything mysterious about an algorithmic approach to composition: most music is highly rule-bound, even formulaic, and so lends itself to that. An early attempt at computer music in the 1980s, the programme called CHORAL devised by computer scientist Kemal Ebcioglu to harmonise chorales in the style of J. S. Bach [1], drew on well formulated principles of harmony and melody that were observed by Bach himself [2].

But it’s one thing to slavishly follow the rules, quite another to come up with original melodies and harmonic progressions that will captivate and even move the listener – especially if the composing is being performed without any human input. Computer scientists have been quite successful in making programmes that can learn from human examples. Early improvisational algorithms such as the jazz-inflected GenJam [3], devised by computer scientist John Biles, and GenBebop, a system created by cognitive scientists Lee Spector and Adam Alpern to produce solos in the style of Charlie Parker [4], gave indifferent results even by their creators’ admission. But Continuator, a programme devised by François Pachet at Sony’s Computer Science Laboratory in Paris, is much more convincing [5]. In a kind of ‘Turing test’ where Continuator alternated with an improvising human pianist and elaborated on his suggestions, expert listeners had trouble telling man and machine apart.

Contrary to common perception, however, improvising is fairly rule-bound too, so it’s not hard to see how human-derived musical material can be plausibly mutated and elaborated in an automated way. Coming up with the raw musical ideas is a much harder task for a computer. Before now, efforts to achieve this have been distinctly underwhelming, sounding like bland pastiche all too evidently shaped by clichéd harmonic progressions and melodic structures.

This is where Iamus’ creators claim to have something new. The algorithm, named after the legendary son of Apollo who could understand the language of birds, is inspired by Darwinian evolution. The computer generates very simple ‘musical genomes’ , rather like little motifs, which are then evolved, mutated and elaborated until they acquire genuine musical content and interest [6]. Genetic and evolutionary algorithms for making music are also not new (see, for example, ref. 7), but Iamus seems capable of dramatic invention: the music is far more than just a succession of transparent variations. Vico and colleagues, in collaboration with composer and pianist Gustavo Díaz-Jerez of the Conservatory of the Basque Country in San Sebastian, recently recorded some of Iamus’s scores with leading musicians, including two orchestral pieces played by the London Symphony Orchestra, for release in September. They broadcast a live performance of two of these pieces from the University of Malaga in July to commemorate the 100th anniversary of Alan Turing’s birth.

The recorded compositions are all in a modernist classical style – full of dissonance, but with hints of harmony and rich textures that might put a listener in mind of composers such as Gyorgy Ligeti and Krzysztof Penderecki. But the same approach can be used for other idioms too, and Vico and colleagues say that it might supply a cheap, convenient way of generating music for commercial purposes.

The willingness of professionals to perform the works marks out Iamus as unique. The LSO’s chairman Lennox Mackenzie was impressed with what it had achieved but felt that the music still fell short of that by good human composers. It struck him as “going nowhere” – a complaint often made of other modernist works – yet ultimately achieving an “epic” quality. Many of the other musicians were pleasantly surprised by the material, and found some of it genuinely expressive.

Which brings us to the initial question. If Iamus can simulate (and thus stimulate) emotionality, is it not just ‘thinking’ in the limited sense of the Turing test but apparently displaying human characteristics?

Here we should heed studies of music cognition which have shown that emotion in music is not some deeply mysterious process but has its own rules and regularities [8]. For example, certain musical structures, such as ‘false trails’ that create and then confound expectations, or judicious injections of dissonance followed by resolution, can elicit emotion quite reliably [9]. This should be no surprise to anyone whose emotions have been helplessly manipulated by formulaic film scores.

What’s more, the involvement of human performers is vital. Music lovers know very well that the same piece can be performed in a dry, unengaged manner or with heart-rending fervour. Good performers achieve expression with a wide range of ‘tricks’, such as subtle distortions of tempo, intonation and timbre [10].

Iamus’ work might therefore be considered to demonstrate the often neglected role of performer and listener in ‘making music’. The nineteenth-century Romantic tradition has fostered a deep-seated belief in the inherent genius of the composer, as though he or she has imbued the very notes with passion. It’s not to deny the undoubted sensitivity and skill of the greatest composers to say that a composition only becomes music in the mind of the listener, through the interaction of the composer’s and the performer’s choices with the wealth of learning and association that even allegedly ‘unskilled’ listeners possess.

This consideration ought also to diminish an inherent prejudice (evident in the critical responses to Iamus so far) against computer-composed music. Neuroscientists Stefan Koelsch and Nikolaus Steinbeis have shown that part of this prejudice is unconscious: the same piece of music may or may not activate parts of the brain associated with ascribing intention to others, depending on whether listeners have been told that the piece was composed by a human or by computer [11]. It’s possible that human performance of computer-made music might at least partly override this obstacle to emotional engagement. But we should also celebrate the way that Iamus, far from threatening our supposedly unique claim to creativity, can put the audience back in the picture as a participant in the creative act.

References

1. Ebcioğlu, K. Proc. 1984 Int. Computer Music Conf., 135-144, held in IRCAM, Paris. Computer Music Association, San Francisco, 1985.
2. Rohrmeier, M. & Cross, I. (2008), in Proc. 10th Int. Conf. on Music Percept. Cognit. (ICMPC 2008), Sapporo, Japan.
3. Biles, J. A., in Proc. 1994 Int. Computer Music Conf., 131-137. International Computer Music Association, San Francisco, 1994.
4. Spector, L. & Alpern, A., in Proc. Twelfth Natl Conf. Artificial Intelligence, AAAI-94, 3-8. AAAI Press/MIT Press, Menlo Park CA and Cambridge MA (1994).
5. Pachet, F., J. New Music Res. 32, 333-341 (2003).
6. Díaz-Jerez, G. Leonardo Music J. 21, 13-14 (2011).
7. MacCallum, R. M., Mauch, M., Burt, A. & Leroi, A. M. Proc. Natl Acad. Sci. USA 10.1073/pnas.1203182109 (2012).
8. Sloboda, J. A. The Musical Mind: The Cognitive Psychology of Music. Clarendon Press, Oxford, 1985.
9. Juslin, P. N. & Sloboda, J. A. (eds). Music and Emotion. Oxford University Press, Oxford, 2001.
10. Meyer, L. B. Emotion and Meaning in Music. University of Chicago Press, Chicago, 1956.
11. Steinbeis, N. & Koelsch, S. Cerebral Cortex 19, 619-623 (2009).