Machine imagines music: HE composes new Wagner music, premiere in Dresden

On the evening of November 20, 2022, the Singakademie Dresden will hold a concert in the Lukaskirche, the program of which will come from an unidentified composer along with Wagner, Schubert, Draeseke and Zemlinsky. Anonymus is not an unknown 19th-century master, but a contemporary machine whose artificial neural network has been trained on music from the Romantic period and which is fine-tuned to the works of Richard Wagner.

Michael Käppler, head of the Singakademie Dresden and conductor, has been running a music project for about a year, for which he was able to win the Technical University of Darmstadt and the Heidelberg AI startup Aleph Alpha as partners. His Meistersinger reloaded project provides a technical and intellectual foundation in addition to sensual and cultural stimulation. The afternoon before the world premiere, a scientific panel discussion from the fields of music, technology, machine learning and mathematics will examine the question of whether machines can compose: three men and a male expert discuss the choir director.

AI commentary on Wagner: Can machines compose?

The origin of the project was the question of how such a problematic composer as Wagner can be interpreted today. Conductor Käppler wanted to “break Wagner’s music in a different way – with an AI commentary on Wagner,” he explained. heise developer. To do this, he turned to various universities and experts and found an enthusiastic comrade-in-arms in Kristian Kersting. Kersting is Professor of Machine Learning and Artificial Intelligence at TU Darmstadt, where he heads the Hessian AI Research Center.

He connected the musician with his TU colleague Prof. Iryna Gurevych, students, Hessian. AI and for research and engineering with Heidelberg’s startup research department Aleph Alphawho then worked on the project as a team. For Aleph Alpha, among others, Jonas Andrulis and AI researcher Koen Oostermijer were involved in building the AI ​​- much of the work done on the Heidelberg AI system. Kersting’s PhD student, Wolfgang Stammer, created a new transformer at the same time.

Aleph Alpha is best known for its multilingual, multimodal core AI model, Luminous, which understands text and images in context. For the Heidelberg AI company’s research team, the goal of the project was to change the modeling of data and relationships in the artificial neural network in such a way that AI, which is otherwise oriented towards conversation or finding information, instead. creates harmony and creates music that also follows a certain style.

Since the structure of music works differently from that of language, adaptations were necessary: ​​among other things, the research team had to expand the way in which words or notes are positioned in relation to each other in such a way that durations and time intervals can be translated mathematically in the language of AI. The work of an entire year and the contributions of many comrades have gone into the composition of approximately three minutes.

Don’t repeat it, but create something new in the Wagnerian style

According to Michael Käppler, they fed a Transformer model with music from the Romantic era, especially piano music. Training was performed on different datasets. Based on YouTube audio and a Google Magenta transcription model, Käppler himself trained a dataset of romantic piano music and called it the Great Romantic Piano Dataset (GRPDS). It contains around 12,000 MIDI (Musical Instrument Digital Interface) tracks with a total length of 1900 hours. During the conversation, the leader added that this data set can be greatly expanded. In addition, he and his partners have also used GiantMIDI-Piano, a dataset available on GitHub from ByteDance – the Chinese internet technology company behind TikTok, among others. The data set provided the signal the team used to continue the work.

In these larger data sets, the model not only learned harmonic rules, but also the intricacies of composition and construction of a masterpiece. Once AI had understood the rules of the music as a whole, the project team brought it closer to the specific style and character of Wagner’s works – to this end, the research team performed fine-tuning of Wagner’s works together with the conductor Käppler. . If the model had trained exclusively with Wagner, he would have learned it by heart and would not have been able to create anything new beyond the familiar pieces (the technical term for this is redundant).

“It’s important to find the balance between a model that parrots and one that only finds specific Wagner music,” Käppler said, explaining the procedure. As an audio example, he showed the Developer’s editorial team one of the many AI compositions that were created in the project and were not processed for the orchestral piece: the first 15 seconds are originally from Hans von Bülow’s Meistersinger Paraphrase, the rest . it’s car work (link leads to audio sample).

“A good number of people have come here”

Transparency is important to him, that is, revealing which parts of the work are human-made and which come from the artificial neural network. In the panel planned before the concert, alongside Käppler and Kersting, empirical musicologist Miriam Akkermann (junior professor at TU Dresden) will talk, among other things, about post-processing. According to Käppler, the project allowed the AI ​​system to compose piano music while the musicians did the orchestration themselves—ie. transposing and changing registers for different instruments: “A good number of people have come in here and it’s important for me to reveal it and with it transparent,” says Käppler (you can listen to an audio teaser of “Meistersinger” here reloaded” – the audio prompt ends after the beep and AI continues the Wagner composition).

A related project would be Beethoven X – not the cryptocurrency portal of the same name, but an elaborate project to complete Beethoven’s tenth symphony using AI by teams of experts from Harvard, Cambridge and Rutgers Universities, sponsored by Deutsche Telekom, among others. The Dresden collaboration for the replenished Meistersinger managed with far fewer human and financial resources. Two to four people mostly worked on the project, and the long-term goal is not only to present a result, but also to reveal the workflow.

Automaton composition: extremely beautiful

The automaton of the composition raises questions: Some may think of ETA Hoffmann’s terrifying Sandman – to which Sigmund Freud attached his definition of the uncanny, as it fascinates us as humans, but also begins to frighten us when mechanical objects have a soul or living beings also appear. similar to the slot machine. Others will dismiss it and question the authenticity of the car’s art.

It should leave no one completely unscathed when a new form of automation is now increasingly penetrating areas previously considered the domain of humans and dealing with feelings, such as expressive composition and art creation. What do composing machines mean for our creativity?

Panel discussion and concert: have your say and make an impact

A panel discussion on 20 November at 15:00 will present the collaborative project between Singakademie Dresden, TU Darmstadt and Aleph Alpha and discuss the boundaries between machine learning and human creativity. The following quartet will discuss whether humans can be replaced as artists – or whether new skills will create a new partnership between man and machine: alongside Michael Käppler and Kristian Kersting, the mathematician Christfried Brödel, who is not only the head of the New Bach Society in Leipzig, but also has a background in contemporary music, as well as Darmstadt AI master’s student and classical percussionist Matthias Lang.

As moderator, empirical musicologist Miriam Akkermann (junior professor in Dresden) brings together the topics of conversation. Aleph Alpha will be replaced by Koen Oostermijer. The panel will take place in the parish hall at Einsteinstraße 2 in Dresden, opposite the Lukaskirche. Entry is free and participation is independent of going to the concert – the round of talks will be recorded and published afterwards.

From 17:00 the evening ends with a concert in Dresden’s Lukaskirche, in which, alongside the works of Franz Schubert, Richard Wagner, Felix Draeseke and Alexander Zemlinsky, the Meistersinger reloaded by “Anonymus” will also premiere. The program can be viewed on the Singakademie website. As far as we know today, the concert will not be broadcast or recorded. If you are interested and can arrange to come to Dresden, you can buy tickets for the concert in advance. The exact program of the musical evening “Art Faith and Faith Art” as well as details on the panel discussion can be found on the website of the Singing Academy Dresden. Tickets for the concert are available from 7 to 20 euros (plus advance booking fee, otherwise at the box office).

(she)