In part one of our AI Futures series, we discussed the looming threats and opportunities around ‘deepfakes’ or style transfers using AI. We spoke to Holly Herndon, a Berlin-based artist who’s been deep in the trenches with AI for many years. We also explored how deepfakes are ushering in a Sampling 2.0 era, and explore how the mistakes of the past have a chance to be rectified for the future. It’s worth reading part one before you continue.
In this section, we’re exploring the impact of AI in the studio. For producers and songwriters, the idea of an autonomous collaborator who makes suggestions for your music or arrangement, helps you write lyrics or simply does the job for you, is generally met with unease. That’s despite the fact AI is already part of some major DAWs including Logic Pro, which uses the tech to automatically detect and create tempo markers as you play. Third party plugins like Google’s Magenta Studio project use machine learning to generate chords, melodies, drum patterns or even a number of bars in the arrangement based on existing MIDI files and some basic parameters set by the user. There are plenty more AI plugins and tools on the market, including Amper, Rhythmiq, Musico – this list of 13 is a good place to start exploring. For this piece, we’re going to focus on iZotope.
iZotope is one of the world’s leading plugin manufacturers, and their tools are used by everyone from Skrillex to Trent Reznor and Just Blaze. In 2014, they began developing a research team that took an unlikely influence for a new ML algorithm to help users mix their music — Facebook.
“Facebook was starting to crack the nut of facial recognition and we became interested in how we could use those techniques for audio applications,” says Jonathan Bailey, iZotope’s Chief Technical Officer. Bailey has been with the company since 2011, back when AI “wasn’t very sexy and buzzy”. The resulting technology — Track Assistant, inside a plugin called Neutron — is based on ML and essentially ‘listens’ to your input, identifies the instrument and makes suggestions around mixing techniques.
“That was really successful for us in a couple of dimensions,” Bailey says. “One, it was a cool technological breakthrough; it was the first example where we really led in the marketing. We branded and positioned this product as powered by machine learning, and that was really interesting to our core audience who represented the nerdier, techier side of the audio community.”