Tomorrowland's virtual festival set a new standard for digital events — here's how they did it
With an IRL event impossible this year, the team behind Belgian mega-festival Tomorrowland created a revolutionary interactive virtual world in under three months. Here’s how they did it
Tomorrowland has never been one for subtleties. As the Belgian festival scaled up from more humble beginnings in 2005, the Boom location became an array of sights and sounds to stimulate all senses, with colossal constructions, epic line-ups and a sense of other-worldliness. Split across two weekends, almost half a million visitors attended the 2019 edition alone.
And, like almost all major festivals on the planet, 2020’s edition was forced to be cancelled. The COVID-19 pandemic has made the gathering of such large crowds impossible, and the Belgium government, like many around the world, announced a nation-wide lockdown through much of March and April. Tomorrowland's cancellation was announced on 15th April, with a statement: “With a lot of pain in our hearts, we have to inform you that Tomorrowland cannot take place in 2020. We understand and support the governmental order that has just been issued.”
Behind the scenes, the team immediately began working on an ambitious virtual festival that resulted in more than 60 artists filming brand new sets on 6x8x8-metre green screens, in four different locations – São Paulo, Sydney, Los Angeles and Boom itself. Working with a creative agency called DogStudio and collaborating with many more, they set about creating an immersive virtual world called Pāpiliōnem, which users could explore by clicking around. It featured eight different stages, all designed from scratch in 3D by Tomorrowland’s own team, as well as companies who’ve helped design virtual sets for the Super Bowl, among others. Artists across the eight stages included David Guetta, Dixon, Charlotte de Witte, NERVO, Stephan Bodzin, Katy Perry, Eric Prydz, NGHTMRE and ANNA.
There were also Inspirational Sessions, a series of 16 talks and panels with the likes of will.i.am, founder of Cirque Du Soleil, Guy Laliberté, and NBA legend Shaquille O’Neill.
The result was a hugely impressive virtual event, conceived, produced and executed during lockdown, with a team of 200 working non-stop around the world. So how did they do it?
"A large grid of infrared reflectors was also installed on the ceiling to allow tracking devices to measure an exact position of each camera head"
“TIME!” replies Tomorrowland press officer Debby Wilmsen when asked what was the most difficult aspect of putting the event, dubbed Tomorrowland Around the World, together. “Everyone really needed to work 24/7 to get everything ready. It was very time consuming to render all the images. Eight stages, 60-plus artists, 16 webinar sessions, extra activities. Also because of COVID-19, everyone was working from home. [It was] also impossible to travel, so we [had to] build four video studios for the artist recordings.”
Before the cameras were set up, the studio facilities got a technical overhaul to get them ready for augmented-reality productions. A large grid of infrared reflectors was also installed on the ceiling to allow tracking devices to measure an exact position of each camera head. All other parameters of the cameras and lenses were transmitted live to servers that recorded the data and rendered a low-resolution version of the virtual world for the camera operators and directors to use. Along with the technical setup, the lighting on the artist changed according to the location of the virtual stage and the time of day during the broadcast.
Translating the green screen on-stage recordings into a moveable, scalable virtual world, where the six 4k cameras could be combined with 38 virtual cameras, for wide sweeping shots of the 280,000 virtual festival-goers, was handled by a company called stYpe. It specialises in virtual and augmented-reality camera tracking and virtual studio effects for live broadcast, and has worked on virtual sets and events for the likes of MTV VMAs, Eurovision Song Contest, Olympic Games, Super Bowl, and many more.
“In live broadcast projects, like for example the Olympic Games, there are very high standards for speed, virtual scene optimisations and reliable camera tracking,” explained stYpe CEO Stype Cajic. “As the shots are done only once, there’s no luxury of repeating them, since it all has to be live. Movie productions, on the other hand, have requirements for high-resolution and photorealistic virtual effects as seen in blockbuster movies. Tomorrowland Around the World had both of these requirements and this meant we had an interesting technical challenge to solve.
“We... decided on using the RedSpy system,” Cajic continues. “It proved to be a great choice, since it operated without problems on dual-frame rate for days in a row, and it surpassed Tomorrowland’s accuracy requirements for 4K shooting. The second thing [we] needed were high-resolution photo-realistic real-time renders. For this purpose, we used our StypeLand Unreal plug-in, which we modified for getting the volumetric lights on the talent, which gave the whole set a more realistic look.”
You might recognise the term ‘Unreal’, from the Unreal Engine — one of the most common 3D real-time creation platforms, used in games like Fortnite, Minecraft, Mortal Kombat and hundreds more. Tomorrowland explain how it worked: “Each stage has a 16-square-kilometre surface with 32,000 trees and plants. The landscapes for each stage have been custom-made to resemble elements of the festival grounds: the Core stage is situated in a forest environment, while the Mainstage has the natural amphitheatre landscape. The digital universe of Tomorrowland Around the World has 10 times more polygons compared to a modern computer game, pushing the limits of the most modern game engines and hardware.”
Finally, a software called Depence by Synconorm had to be re-coded by the developers in order to support real-world footage, and create realistic lighting on the DJs based on each virtual stage’s lighting, fireworks, FX and lasers. Crowd cheers and noise were added too, for which all Tomorrowland employees took part in the audio recordings.
"I can proudly say that we are setting new standards for web-based online music experiences, pushing the boundaries of the latest technology that is available"
Once the sets were filmed, over 300TB of footage was collected — 329,853,597,770,307 bytes to be exact. Tomorrowland and DogStudio began the process of turning the footage and renders into a working and interfaced world for users to navigate around on any browser, and on older, less-powerful devices: “Our biggest challenge — besides being an obvious enormous technical challenge — is making sure festival visitors will be able to feel they are being part of something larger than their computer and their internet connection,” DogStudio CEO and Creative Director Henry Daubrez explained. “I can proudly say that we are setting new standards for web-based online music experiences, pushing the boundaries of the latest technology that is available, but on the other hand making sure that the platform is even working on a device that is a couple of years old.”
As virtual festivals and live-streamed music events continue to find their feet amidst the unprecedented global disruption, there've been streams from some incredible locations and unconventional interfacings like Club Quarantine, Junction 2 Virtual and DJ Mag’s own Top 100 Virtual Festival. But where the vast majority of those experimenting with the new virtual electronic music performance landscape let users access streams for free, or a nominal amount, Tomorrowland sold tickets starting at €20 to access the event ‘live’. There were also multi-ticket options for those watching with friends around the world, and a Relive ticket for €12.50, letting users access the world of Pāpiliōnem once the event had ended to re-watch the sets of their choice.
Tomorrowland says they had over 1 million people watching over the weekend of 25th to 26th July. The Belgian festival also released audio recordings of some sets to Apple Music subscribers. Though it hasn’t been confirmed, it’s likely the sets will eventually end up on their YouTube channel once the Relive access ends, sometime after 12th August.
Tommorrowland’s achievement from a technical perspective is remarkable, regardless of timescale. To do it all in three months in the middle of a pandemic is close to unbelievable. And while it took a team of 200 people, access to the gaming, film, and event industry’s most cutting-edge technology, and tens of render engines operating 24/7 for four weeks to process the data, it serves as an example of the scale of what’s possible as events companies, promoters and artists continue to attempt to adapt to an uncertain future. Of course, few will be able to match Tomorrowland for scale, but it’s encouraging that over a million people paid for access to the festival, at a time when the industry needs fan support more than ever.
The virtual festival isn’t just for 2020 — even if the real event goes ahead next year, Wilmsen says the virtual version will return. “We think this can be a very nice thing next to the real festival. It gives many people the opportunity to experience the festival. People who are not able to come to Belgium [because of] age, visa, tickets, health; the digital festival is open for everyone.”
If you missed Tomorrowland's virtual festival you can watch the after-movie to see the results below.