Interview: How Fortnite Inspired Building a Better Virtual Concert in Sky: Children of the Light

With the curtains closing on the season of Aurora within Sky: Children of the Light and its interactive concert, we had the chance to sit down with creative director of thatgamecompany, Jenova Chen, to talk about it. We explore how this awesome experiential collaboration came to be in the first place, what it was like to select the songs, and talk about what's in store for the future of Sky and if more concerts might be on the horizon. Let's dive in!

Push Square: So, talk about how this collaboration with Aurora came about. How'd you end up making this incredible interactive concert experience happen in the first place?

Jenova Chen: In [our previous game] Journey, we had a really awesome credit song, sung by Hollywood singers. So we were like "we better have someone who has that spiritual power behind it." So we just walked up to Aurora, she was doing a concert in New York. My wife and I were really big fans of hers, so we just decided "let's walk up to her and see if she'd be willing to help us!"

And she said "Oh! I played Journey four times. I did everything!"

It was so amazing that this is the way the industry can work sometimes, it just opens doors. Now, we're back after three years, and we did a whole concert with her.

Each of your games has a strong affinity for music, with a large amount of their popularity focused on their soundtracks. With Sky, literally putting a concert experience inside the game seems like a natural fit. Was that a conscious decision to lean into that musical angle?

You know, at the beginning of Sky, there's a line when you open the game, that says "Music is more than half of the experience." Basically, we're trying to tell you to put on your headset, right? [laughs] George Lucas used to say that music is half of cinema, but to me, it's more than half. Music is the most effective and efficient language of feelings. It's more effective than words. Much more nuanced than any language you can find. That's why when we watch cinema, a lot of times we don't know how to feel, and we rely on the music to give us a hint. How are we supposed to feel? Imagine any scary, horror game without musical cues.

Ultimately, why music is such an important part of [thatgamecompany's] games is that our focus is to push intensity and also expand the type of emotions that games can communicate. We like to create new feelings. Music has become one of the most reliable instruments in our orchestra.

So this time, when we started on the Aurora season, we didn't just build a concert, we built a music theater. If you play the season, you'll experience the narrative of each of the songs that Aurora personally wrote. We wanted to channel the soul of the artist, so we worked with her, and she told us what she saw in her mind when she wrote each of these songs. And we would turn each of these descriptions into interactive dances.

Something like synaesthesia!

Yes! You should check out that first song, 'Runaway'. Many people cry over that song. Her music and her vision working together like that, many people just feel instantly captivated by it. I sound like such a salesperson here [laughs].

We really want our players to dance to the music. A lot of the time with virtual concerts, the first thing you think about is standing in like a mosh pit, jumping. But that's all you do. What we want you to do is to dance with the music. To swim within the narrative of the songs.

The idea for the amphitheater as the staging device is really interesting too. When you consider most virtual concert spaces so far, it's just like a group of a hundred or a thousand people, kind of standing around in a random field in Fortnite.

It's funny, in Fortnite, the best thing you do is shooting, but you can't shoot in the concerts. Whereas in Sky, the best thing you do is give each other a hug, and hold hands. And you can do that in the concerts. And I feel like we're better suited to the concert because concerts make people feel emotions.

Fortnite is massive, so tons of people will show up, but the gameplay is incidental to that, and the gameplay doesn't really suit it, as opposed to Sky, where it feels like it was made for it from the ground up. It reminds me almost of 1 vs. 100 on the Xbox 360 if you remember that.

Yes! Right! [laughs]

When you are there, you can actually chat with the people who sit on your bench. A lot of people are like "don't ever do that, people are really mean!" But we launched it on PlayStation and so far people have been really nice. The whole thing, these people get to experience a full concert, and it's a really strong bonding experience I'd say. A lot of people will stay there after and make friends.

Let’s change gears. I was curious about the logistical process of getting the concert put together. When Sky launched on Apple TV back in 2019, was the concert something you were already planning on? Was any groundwork already laid out?

So the game has been around for three years, and when we first built the game, it's really an 8-player game, an 8-player co-op game. But, starting around a year and a half ago I think, I saw that first Fortnite concert, and that was really inspiring to me. I had never thought about bringing something like that from the real world into a virtual fantasy land. And so, at that point, I really actually did enjoy the Fortnite concert, and I started to think about, is this possible? It was very inspiring. Journey and Sky were focused on bringing people together. Journey is about bringing two strangers together and creating a genuine human connection. Whereas Sky is 8 players. You can have a friend circle, a family, hanging out together. But when I saw the concert, I’d never thought you could bring tens of thousands of people together in one go! That was something I'd never even dreamed about attempting, but after the Fortnite concert, it was like "maybe I can give that a try."

One thing that stood out to me [about the Fortnite concert]: I never felt like Travis Scott was really there. He never looked at me. I don't know his soul is there. So I wanted to build [an environment] where you can feel the presence of the artist. Another thing, I went to a Taylor Swift concert, and I was in the stadium with 30,000 people, and it definitely felt like I was a part of something bigger. These are the two things that I was like, "I think we can push the boundary on how a game makes you feel."

First thing, how can we make it feel like [Aurora's] soul is present? Well, that's easy, make it so you can interact with her. You can interact with her character; you can know what she's like, her personality. The second part was the hard one, the 10,000 players all in the same session. So then we spent about 6 months working on this technology. We would [project] every single 8-player game room into a centralised server, so we can see every single player around the world playing the first level of Sky in real-time. It was like merging thousands of servers into one server. So we have this data, but can we render 10,000 players in one go? The Switch is pretty old hardware at this point, and we have some pretty terrible Android phones, so how can we render several thousand individually animated, moving characters? We spent a season going underwater and pushed our tech team to render seven thousand fish on the same screen. And have them blocking and animating. Our player would feel like "why are you guys doing this, you don't need this."

But we’re thinking "how can we use this tech to drive the data of the players?" So when testing this, we could see over a thousand people on the first level, but that was more people in one go than I'd ever even seen in an MMO. And then, we start to see what happens at the home area [Sky's main hub], where there's usually like 10,000 people just standing around changing clothing. So we tried rendering them as like jellyfish, since it's so overcrowded that you can't really see anything. So then we started to think about how to optimise the rendering of the players. And then we spent one season optimising the stadium.

In order to enjoy a concert as someone who doesn't play games, the best intuitive thing to do is take something from the real world and bring it into the virtual world, that way they don't need a tutorial to use it. So we designed the stadium around the Rose Bowl in Pasadena. And we designed different activities and merchandise places, and whatnot so people can spend some time there before the show.

But initially, we didn't know how to render so many players. We wanted each player to look different, but it's instance-rendering, so they have to look the same. But eventually, we could pass the player's internet data along to an instance, so their cape colour, their body weight and height, would influence each instance to represent how their characters looked in the regular game.

The whole thing took us about a year and a quarter to propel the technology. Our players just thought we were building technology for the season.

What was the process for song selection? Did you guys pick? Did Aurora get to choose, was it collaborative?

So on an Apple Music interview, on the Travis Mill show, Travis asked her what the most difficult part of working with [us] on this concert was, and she said it was for her to pick 7 songs out of her entire portfolio. When she picked the songs, I asked her if she could give us some inspiration. So she gave us descriptors.

Like with 'Runaway,' she said "It's a small boy, I feel like we have to help him over the ocean. We have to help him get home. He is lost. Maybe we can show him that he can belong with us. Have a home with us."

And we also asked Aurora, if a player were to come to your actual concert, what do you expect them to do? So she showed me this emote, and I said "what is this?" And she said it was how deaf people clap because deaf people go to her concerts, which I was pretty shocked to hear.

So when she picked these songs... We actually took one out because we ran out of time, but she didn't pick 'Cure For Me', which at the time was her most popular song. But she wanted to give us a range of all kinds of emotions, ups, and downs.

Her favourite song is 'The Seed' which I actually agree with. But there’s a line in that that says, "When the river was poisoned, when the last tree has fallen, you cannot eat money." And it's based on one of the last American Indian chiefs when the colonists drove them out of their land, he said this. [The quote in question is a Cree Indian prophecy]. But when she made the song, it was from the perspective of a seed from a tree. And it's a very strong environmental perspective, so we worked that song with my own sensibilities. And look at Flower. If you've played Flower, that game is about that tension between humans and nature. So I turned her songs into this narrative about how nature views humanity. And then the player would be playing as nature. They're the fish, the bird, the manta ray. So the player is essentially a part of the concert, and they experience humanity from a third-person perspective. Humanity is something else. But that's how Aurora picked her songs, and then her song choices influenced our narrative decisions for the concert, and it all came together in the end.

This season just ended, but now that you've spent all this time making this technology, and it's gone off so well with the Aurora concert, do you intend on doing more with it? Either with Aurora again, or with others?

Definitely! And it's not just for concerts either. With the technology we have now, we can have a thousand people anywhere in Sky. So we are heavily brainstorming, how are we gonna use this tech in the right place? Because you don't want like 200 people sitting in front of you when danger is happening, it's not scary anymore. But you kind of want places where, if there are a thousand people there, you actually want them to be there. On PlayStation, if we wanted to, we could render like 40,000 players. Easily. So we're trying to figure out how we can actually use the number of players to give them the best experience. It's not just the more the merrier. But even with traditional games, like MMOs, you only have like what, 40 people in a dungeon raid?

But when you have 4,000 or 10,000, it's kind of a different beast. How do you even design an experience like that when they can all interact with each other at one time?

Even comparatively, that's the equivalent of going from a slightly crowded restaurant to something like PAX!

Yeah! We were joking about that actually. We wanna do a PAX or like, "Skycon" in the game! But yeah, that's the area we're looking to. We're really excited that Sky is finally on PlayStation, with the 4K and 60 FPS, it just looks much better than any mobile version of the game.

So, I'm like, okay, if we're on PlayStation and we can put 40,000 people on screen, what can we do with that? So there's this big explosion at the end of the concert, where everyone comes together and gets pushed outwards, and that came about as kind of an accident. Our designer was like "well what can we do with 4,000 people, maybe just a hold and release?"

If you're playing it, you can feel like she's pushing you at that moment, but there are so many possibilities to do with so many players. 1 vs. 100? We've got 1 vs. 1000 here!

Time to bring the show back!

Honestly, I wish I could tell your our plan, but the truth is we're still trying to figure it out. It's very hard to test. During development, we can only get like a couple of hundred people. We can never predict what 4,000 people would look like. Even the level we designed, we can put 2,000 jellyfish in a cave, but it's completely packed, you can't see anything. But that's the fun of making games with crowds.

When you consider a massive game, a large volume of players, it's like 200 maybe. And that's probably excessive. MAG on the PS3, that was 256, and that's one of the bigger ones.

As a cross-platform, we have a pretty high concurrent player count, and during the first concert, we had 1.6 million players, so we had 400 of these stadiums of 4,000 running in parallel.

Well, now the next step is to get those 1.6 million players all in one instance!

Yeah! This technology makes me think about stuff like that. But we're having fun [making it] and hopefully, you'll find it fun as well!

We'd like to thank Jenova for taking the time to talk to us about thatgamecompany's newest title. Have you played Sky? Did you participate in one of the Aurora concerts? Sound off in the comments section below.