The Art of Transmedia Storytelling, with Chris Ferriter

Jordan: 00:00

Welcome to Playmakers, the game industry podcast. Whether you work at a studio, publisher, service provider, or startup, this is the podcast that will give you all the information and entertainment you need to succeed in the game industry. Who am I? Just your friendly neighborhood veteran designer and producer, Jordan Blackman. In each episode of Playmakers, I go to work uncovering insights, tactics, and know-how from a wide range of game industry luminaries. My goal: to help you win the game of making games. Are you ready? Then let's begin.

Welcome back to Playmakers. I guess I just said that in the intro. Now it's like, now me. And that was like, then me. In any case, we've got a great episode for you today. Our guest is an award-winning producer and designer whose career spans both the motion picture and video game industries. He worked as a visual effects artist on Disney's Honey, I Shrunk the Audience and also worked on several movies before transitioning to the game industry, where he produced like 50 games for companies like THQ, Midway, Ubisoft, and EA Sports. Currently, this guest is the—alright, I'm just gonna say his name. This is Chris Ferriter. Chris actually managed me in one of my jobs.

So he was my boss way back in: 2007

And we'll get into what that means. If you're not sure what that means, but we'll get into that. So Halon has collaborated with amazing products and they've worked on Star Wars, Avatar, Transformers, Spider-Man, Star Trek, Planet of the Apes, The Mandalorian, and... you get the idea. These are total badasses. And Chris is the CEO over there who runs the whole operation. So Chris is an absolute expert at bringing cinematic storytelling and cutting-edge production techniques to storytelling in games and in film. So that's what we're going to talk about.

In this episode, we get into his journey back and forth between film and video games and what processes he and his team recommend and use for developing cinematics, what recommendations he has for bringing more cinematic elements to smaller games that maybe can't afford the big-budget cinematic kind of stuff. And we also get into some of these really hot new real-time virtual production techniques and how that is changing the game for cinematics. And this is stuff you definitely want to know and want to be aware of because we also talk about how this is going to be affecting the development of storytelling in games over the next five to 10 years.

So if you're excited about the way storytelling and games come together and excited about learning the experience of one of the very, very best who has lived both sides of that, then you are going to love this interview. Ladies and gentlemen, this is going to be with Chris Ferriter. Now, before we get into the interview, I just want to remind you that if you love the show, if you like what we're doing, if you appreciate the unique approach we have to bringing value to the game industry, then I would so much appreciate it if you would subscribe and write us a review. The reviews and the subscriptions do a lot to letting me know what you think, that we're progressing, that people are listening, that people like this show.

So I very much appreciate it. And if you have any feedback for me about what you want to see on the show, hear on the show, then you can send me an email: jordan@brightblack.co. And I'd love to hear about it. So with that said, let us dive into the interview. Ladies and gentlemen, Chris Ferriter.

Chris, thanks for coming on Playmakers. It's good to have you here.

Chris: 04:18

Thanks for having me.

Jordan: 04:19

I always enjoy getting the chance to connect and to catch up with you. So, we're doing it live, like you said. Do it live. So, one thing when I think about you that I find remarkable is that you've gone from, I think you started in film, then you went to games, then you went to cinematics for games, and now you're doing a lot of real-time movie production for the film industry. So, I don't think I know anyone who's really had that ping-pong trajectory between these two massive entertainment industries. How did all that happen for you?

Chris: 04:48

Luck more than anything. I think I've been really fortunate. I think to be in the right place at the right time a few times in my life. My real start in, I guess, the industry would have been in graphic design. I started as an artist when I was a teenager, and that's what I did as a kid. I drew. That's all I ever did. I sat on the ground, I drew, I painted. And sometime when I was a teenager, I got a job designing t-shirts and flyers for raves, actually. That's what it was.

Jordan: 05:18

Are we talking 12, 16, 18?

Chris: 05:22

I was 13 years old.

Jordan: 05:24

There we go.

Chris: 05:24

Years old, yeah. That's it. And I couldn't even go to these. But I grew up in a household where a lot of people in my family were in the film industry in some way, shape, or form.

Jordan: 05:34

I remember seeing at your house, some old, like reel-to-reel movie... What's that?

Chris: 05:39

Moviola.

Jordan: 05:39

Moviola?

Chris: 05:40

Moviola was an old editing system before nonlinear editing systems came about. Nowadays, everything's edited on Avid, but back in the day, they used to actually cut film.

Jordan: 05:50

This is like a giant metal contraption that was to help you cut the film in the right place.

Chris: 05:55

That's right. Yeah. My mom was a negative cutter, which is a job that really doesn't exist anymore.

Jordan: 06:01

You grew up in the industry.

Chris: 06:02

I grew up around that, which I think made a big difference for me, just in terms of knowing what opportunities were out there. I remember I was having a conversation with some folks at BAFTA about a year ago, and they're very active in putting together these programs to showcase film careers for inner-city kids. And I thought that was... That's really great because there are these kids that go to school a few miles from here and don't know anything about careers in the film industry outside of actor, director, producer, when in fact, there's an entire industry here that employs tens of thousands of people. I feel just growing up around that gave me an awareness of different avenues that you could pursue, that things were possible.

Jordan: 06:47

And did you start off in film or did you start off in games?

Chris: 06:50

Yeah, I started in film. My first job was working for my dad, and my dad was an optical cameraman. He did optical visual effects and then later digital visual effects. So one of my first jobs was running a black-and-white film processor developer and running color tests on optical printers. And this is at a pretty young age. So I was around film. I was around all of this equipment, but this was when everything was done in a way that's much more manual than it is today. Because of my previous experience with art and my love for art, when digital visual effects came onto the scene, and Silicon Graphics workstations were around, I would spend my time at night on these SGIs, learning different software packages and learning what kind of artwork I could do in there.

And I learned Cineon, and I learned Flame, and eventually was able to parlay that into a career as a visual effects artist, and I worked on a number of different films. One kind of funny story I tell from time to time is one of my first jobs was on Honey, I Shrunk the Audience that ran at Disneyland.

Jordan: 08:05

Yeah, I went to that.

Chris: 08:06

It might still run, I'm not really sure, but I hand-painted a lot of that snot when the dog sneezes. But yeah, that was my start—visual effects. And I made my way from there into the game industry, oddly enough, because visual effects was feast or famine. You would get a job, you would make a lot of money for a short period of time, and then you would be without work while you searched for another gig. And it was hard on me as a young man. I didn't have enough money to be able to go four weeks without work. And a friend of mine had a job testing games at THQ. He said, "You should come interview here."

Jordan: 08:42

That was in Calabasas?

Chris: 08:44

It was in Calabasas, yeah. Right down the street from where you were, as a matter of fact.

Jordan: 00:08:48

Right.

Chris: 08:48

So I went in, I had an interview, and I got the job. The next week, I was testing games. And that kind of kicked off a career in the game industry that went for 16, 17, 18 years.

Jordan: 09:00

Yeah, because when we connected, you were like a senior producer at Ubisoft. So you'd come a long way from a tester at THQ—different city, different company.

Chris: 09:10

Yeah, I'd been to many different companies by that point. I pretty rapidly got into production at THQ. So, I tested for a while, but I was ambitious, and I took on a lot of tasks that were outside of what I was required to. And I got the attention of Rob Baumsteiger, one of the producers there, and Scott Travis, who sadly passed away. They gave me a chance to go into production, so I took a job as an assistant producer.

Jordan: 09:35

So for the listeners, I just want to point out how important this is—that Chris got the attention of people who saw in him his ability to do more, to take on more, and that's how he was elevated. And I know there are a lot of listeners who are wondering how they can be elevated or how they can break into the kind of position they want. I think that's a great highlight. I just want to make sure to put a stake in the ground there.

Chris: 09:54

I think it's worth emphasizing because these days I employ a lot of people. Both industries—film, games—these are very desirable jobs that people are competing for. It doesn't hurt to have a college education or an advanced degree, but oftentimes it's being able to just prove to somebody that you are reliable, that you can take on the job. I, in my position now, value people I can trust more than anything. And if somebody shows me they can do a job, I will give them that opportunity.

Jordan: 10:27

I think a lot of people believe, and I've heard this from people, that, "Oh, the industry has changed, and now it's all about the degree. People are looking for you to have gone and gotten a game degree or whatever," but that's really not true. Once you get that foot in the door—it can still be tough—but once you get that foot in the door, wherever you do it, then you have the chance to get people's attention, to prove yourself, and to show you are trustworthy, responsible, effective.

Chris: 10:52

That's not to put down going through the experience of getting a degree. But I think that one of the most valuable aspects of that is if you are going through that system, that game design program, you're working with a lot of people that are also focused on a similar goal. And you're building your network. You're starting early building your network. And some of those people will go on to have jobs in the game industry. They will remember the people that they worked with that did great work, and they'll pull them along. So I don't want to be down on the university system because I think there's a lot of value there.

Jordan: 11:27

100%.

Chris: 11:27

But ultimately that's what it takes—simply getting that foot in the door.

Jordan: 11:32

I agree completely. That's great advice. You worked in the game industry, I think you said 16, 17 years at studios and publishing companies doing mostly production. What brought you back to getting into the world of game cinematics and video production? How did you make that transition—kind of back, in some ways back home?

Chris: 11:51

Jordan, you and I worked together on CSI, and CSI I thought was a very interesting project in the sense that we did our best to bridge that gap between what was happening in the TV space and what was happening in the game space. And we were limited in a lot of ways in what we could do, but our goal was always to tell a story that gave value to that audience. And at the time, CSI was the biggest show in the world, so it was a very big audience. And I wanted, and you wanted, and Tony Van wanted to bring that value to that audience in a way that, a lot of times, games that are based on film and TV licenses did not do. I always felt like they were somewhat parasitic in the sense that the goal was: "We've got a license, let's get this game out the door day and date with the film so we piggyback off of the marketing." A lot of times the game was shovelware.

Jordan: 12:46

And it would be like, "Okay, we're going to take those characters and we'll just put them into a shooter, or we'll put them into a driving game," or whatever, as opposed to creating a bespoke experience that fits those stories.

Chris: 12:57

Absolutely. There was no love behind it. And I became really passionate about this idea of what I guess was referred to then as transmedia—this idea about telling stories across multiple different publishing verticals. I have believed since then that one of my callings in this industry is to understand how to tell better stories, to create better stories, and look at it from the point of view of: What brings value to the audience? Not, "Let's get a story here on TV, let's get a story here on games, and let's get a story here in your PR campaigns." I've always believed that these things should be much more integrated. It must have been 2016, I got a job here at Halon to manage their game cinematics division.

And I was very fascinated by that because Halon was a previs company. And previs existed to tell better stories. The whole purpose of previs is to take a script, work with the director, animate out shots or sequences in that film, and iterate on it until you find that best expression of that director's vision.

Jordan: 14:09

So I just want to make sure I have this right—Halon basically had a division that was doing cinematics for games, but that wasn't the main thing that Halon was doing. The main thing was the previs for the film industry.

Chris: 14:20

Yes, that historically has been the bread and butter of this company. The thing that really interested me is you had all of this talent that had worked with every director in Hollywood, over 200 films, working side by side with that director to really craft better stories. And they were interested in doing game cinematics. And I have believed that there was a lot of room in game cinematics to tell better stories, to approach it the way film gets approached as a visual, linear storytelling medium.

Jordan: 14:52

Well, I saw that the way that you and Tony approached story in CSI, I never forgot that. For example, how all the CSI stories were created with the show writers before anything else was done. That was step one.

Chris: 15:05

Yeah. Story was first. Story was first, and that's where I think it belongs. For years, though, cinematics in games were not approached first. And there's a fair argument that they shouldn't be in that medium. That medium is an interactive medium, and you want to approach the interactive first. But there were a lot of examples of games that looked at cutscenes as something they should not invest in. They were a necessary evil that most players were going to skip right through. You got them in there to bridge the gap between levels.

Jordan: 15:38

Give you something for the back of the box. That's interesting because I think, at least for the big products now, it's become so important, such a vital piece of the experience.

Chris: 15:46

Absolutely. I felt as though you had tools that you were bringing into your game—tools that could build a better story, that could create a more engaging experience. And I think it's great to see the industry has come around to understanding that. The production quality of cinematics these days is just absolutely amazing.

Jordan: 16:06

So I'd love to give the audience a taste of the process for big-budget cinematics, like how they are conceived, when in the process, and what are some of the key production technologies. I know this is a big question, so you can take it where you will—just the major chunks of the process, so people have some understanding of what goes into making Red Dead Redemption or Fallout content or things like this.

Chris: 16:32

Yeah, I mean, it's a huge undertaking. If you take games in that echelon, you're looking at producing the equivalent of three animated features in content. If you look at the amount of work that goes into the creation of an animated feature, you can get some idea—this is a huge endeavor.

So, it will be tough to really touch on everything. And I think that also, different companies do approach it very differently, and different genres approach it very differently. But I can tell you the way that we tend to approach it. We typically work with the game designers or the game production team to understand what story it is that we're telling.

Sometimes they have a script. Sometimes they have previs. Sometimes they just have nothing but an idea. So let's take that example, where there's nothing that is existing. We would work with them to outline a script. We would take that forward to storyboards, where we would start to storyboard out all of the major sequences, and then we would move that forward to previs. It would likely be a very rough previs, where we're just basically doing a layout pass.

We're understanding the cameras. We're understanding what characters are going to be in each scene. From there, you can start to really effectively plan your motion capture sessions. Motion capture sessions are not on the scale of a live-action shoot for a film, but it is a fairly large endeavor. You're working with casting to sometimes bring in A-list actors. You've got sound, you've got facial cameras, you've got body cameras, systems, and a motion capture volume that may have 80 cameras that are all positioned around the volume.

Jordan: 18:21

The volume is the object?

Chris: 18:23

The volume would be the stage or the area that's being captured. The object—the actor in that situation—would be wearing typically a motion capture suit. That suit will have tracking markers on it, and those cameras are tracking the position of each individual marker. Those get then targeted to a model that is rigged to be animated in a specific way. You can apply the motion of those individual markers to that model and produce very realistic human motion on a 3D character.

Jordan: 18:57

What's state of the art in terms of the facial animations and expressions these days?

Chris: 19:02

I mean, I don't want to go too far into promoting anybody specific, but there are definitely technologies out there with ultra-light head-mounted rigs that have multiple cameras tracking your facial movements with a very high level of fidelity. There are animation rigs that can reproduce that data at a very high level of fidelity to the point where we are very rapidly approaching real-time photorealistic facial animation.

Jordan: 19:32

Is LIDAR involved in any of this, or is that not really part of it?

Chris: 19:35

LIDAR is certainly involved in the process. We use LIDAR typically for film more than we would for games. These days, we're doing a lot of work in the virtual production and virtual art department space, where we're building photorealistic environments. Oftentimes, those environments are captured with a combination of photogrammetry and LIDAR to produce a very, very realistic 3D model.

Jordan: 20:03

So what I think you're saying, I just want to make sure, tell me if I've got this right. When you're shooting mostly for virtual productions, you'll get a very detailed LIDAR map of a real place so that you can make a virtual version and shoot scenes or present scenes in that environment.

Chris: 20:19

That's exactly right. Yeah. That'll go into a photogrammetry pipeline. Photogrammetry is a process where you take a number of photographs of an object, and you'll put that through a program like RealityCapture, which will take all those different photographs and piece them together.

Jordan: 20:38

That's a great name, RealityCapture.

Chris: 20:41

It'll piece it together into a 3D model that will contain all of your texture and lighting data. Then that can go through a pipeline where you "delight" it, remove that lighting data, and you can put it into a real-time game engine to light and move around that environment in real time.

Jordan: 20:59

That's amazing.

Chris: 21:00

Basically turning a photograph into an object. It's amazing.

Jordan: 21:03

I want to talk a little bit more about the virtual production for film, but before we get there, I have a couple more questions about the game side. One of the interviews we did on Playmakers was with Ian Milham, who actually works on The Mandalorian. So maybe you guys crossed paths—I know that Halon worked on that. Before that, he was working on Hardline, and he worked on Dead Space. One of the things he talked about a lot in our interview was the ‘Ripomatic’ early on in the production of games. And I was curious if that is something that people ever involve Halon in, or if you think there's an opportunity there.

Chris: 21:36

It's something that we've done occasionally here at Halon. It's something we did a lot of at EA.

Jordan: 21:40

At EA. Yeah.

Chris: 21:41

We used Ripomatics quite a bit. But it is all in service of trying to pull together the story as early as possible in the process. So all of these things are just different tools that you're going to bring to bear to solve the problem you're trying to solve. What are the creative targets that we're going after? And that can be visual targets; those can be story targets. But the Ripomatic is a great tool. It's a way to very rapidly take a bunch of items from a bunch of different places, almost like a collage, throw it together, and tell your story.

Jordan: 22:13

Right, or try to communicate this sort of vibe that when you're developing something new, especially a new IP, it's so hard to get the team to kind of grok the overall vibe. It even sounds abstract when I say it.

Chris: 22:26

But it's one of the biggest challenges. It's one of the biggest uses for previs in films. It's one of the biggest uses for previs in games, but you want your team to coalesce around an idea. You want to have this shared understanding of the targets that you're moving towards.

Jordan: 22:42

Totally. So last question on the game side, I'm curious, based on these experiences, which are generally for very large budget, triple-A kinds of games and for movies that obviously have a massive budget for the cinematics, what are some of the ways that you think smaller developers, who maybe can't hire out cinematics, can bring some of this process or thoughtfulness to their stories or their games to create a more cinematic experience?

Chris: 23:10

I think there's definitely application for all of this in any budget range. We're talking a lot about the triple-A games, but there is no reason that any game team can't start to adopt the same type of processes that you would adopt to find the story on a triple-A game or find the story in a film or TV show. You may do it on a smaller scale, but there's no reason that you wouldn't do it. I would say the one thing that could be helpful for smaller development teams is to go in with a good understanding of what tools are available. One thing that I have seen—and this may not be true these days—I think indie developers have become much more sophisticated.

Jordan: 23:55

Indie now applies to such a broad range of activity.

Chris: 23:59

Yeah, but I think that, in my experience, I've seen teams that have rushed ahead without understanding what tools are available. And sometimes it's difficult to fully understand what you don’t know. But there's a lot of value that can be gained from understanding how stories are told in film because it's a different medium.

It's a linear medium, and it's all about telling your story through pictures. And that sounds really, really obvious, right? But when you get down to it, it's a very different medium than telling your story through an interactive experience. So there's a lot of value in understanding how a film is broken up into shots, what story you're trying to tell in every shot. Learn about lensing. Learn about the types of shots that are strung together to form a sequence. Learn about the cinematic language. And there's so much information available online, so many tutorials that you can get into on YouTube that will talk about cinematic language, that will talk about the tools that a filmmaker brings to bear when they're trying to tell a story.

Jordan: 25:12

I've seen some good YouTube videos too where they deconstruct shot by shot a scene. You know what's helpful for me? Learning photography, just like becoming a photographer. Getting a camera with interchangeable lenses allowed me to speak with much more granularity towards that piece.

Chris: 25:27

A film is a sequence of images, and every shot should tell a story. It isn't that different. The way that you would compose a shot—I mean, obviously there are differences, right? But in theory, the way that you would compose a shot is not that different from the way that you would compose a photograph. And game engines, in a pretty accurate way, replicate real-life lenses.

Jordan: 25:50

Change the field of view.

Chris: 25:51

Absolutely. I mean, this is what we do in virtual production, right? We create digital versions of real-life objects that filmmakers can interact with. A lot of

Jordan: 26:01

Games now will let you, like No Man's Sky or Ghost of Tsushima. You can just pause it at any time, move the camera around, change the field of view, and change the depth of field. That’s a good way to experiment, too. I’ve never spent too much time doing that, but it’s another way to get a feel for it.

Chris: 26:18

Yeah, and I don’t know, this may sound terrible, but copy from the best. Look at how other people are telling similar stories. Look at how filmmakers have told stories over the last hundred years and start by just replicating it.

Jordan: 26:36

I think that’s great advice for learning the mechanics of cinematography and the interaction between what you see and the emotions that are created. If the camera is further away from the player character in third-person versus over the shoulder, what does that imply about the character? And what does that make the player feel? What about the story? And this is, like I said, something I’ve learned a lot from you. In terms of story development, if someone’s out there and they’ve got an idea for a game, how do they make sure they have a story that’s going to work? That’s going to be a good experience for their audience?

Chris: 27:09

The first thing is real obvious: be open to feedback. There’s a lot of value in being passionate about an idea and really sticking to your guns, but there’s a lot of value in listening to other people and listening to criticisms—without being precious about your ideas. Maybe that’s an obvious piece of advice, but that's something that I have seen contribute to-failure of telling a good story many times in my career.

Jordan: 27:38

I think it's great advice. And it’s the kind of thing that I think, how do I put this? I know that I need to hear multiple times throughout my life, right? Like, it’s not like I just hear that once and I’m done. I need to be reminded of that. So I think it’s great.

Chris: 27:51

My goal is to tell the best story. It’s not necessarily to tell my story the way I first conceive it. I try my best to be open to critical feedback and to seek out critical feedback because I want people to challenge the assumptions I have.

If there’s a benefit to be gained, I’m not precious about where that benefit comes from. If someone brings a good idea to the table, take that idea and run with it. At the same time, understand what you’re trying to achieve and be willing to not implement everybody’s feedback. You can’t please everyone, and there has to be a creative core to any story. So be that creative core, but take feedback wherever you can get it.

Jordan: 28:32

I love that. Yeah, I like to think of the creative core as the lens through which all the feedback should be filtered. So instead of looking at whether it’s your idea or somebody else’s, it’s “Does this idea serve the creative core?” That’s a really great way to put it. Let’s get into what’s changing in the world of virtual production, cause I know you’ve told me some amazing things about how things are changing and what’s happening right now. I’d love for you to share that with the audience.

Chris: 28:56

Sure. Virtual production is, quite simply, utilizing a different production workflow that is, I think, centered around the use of a real-time engine as the production hub. Unreal Engine is the engine we tend to use. In theory, it could be any engine—I know The Lion King was built around a Unity workflow.

But Unreal Engine is the dominant one in this space. It’s about utilizing that real-time engine to make better decisions earlier on about the film you’re making. And it’s exciting to me because, in my career, I got to see the transition from optical effects to digital visual effects. What came with that was this idea that you could now make decisions much later. You could fix things in post. You didn’t have to be as clear about what you were trying to achieve beforehand because you could always fix it.

Jordan: 29:53

Now you could add new creatures to Star Wars in the anniversary edition or whatever that was.

Chris: 30:00

That’s right. But this, I think, is very fascinating because it represents a bit of a reversal in that workflow. Now, we’re starting to make more of those decisions in pre-production. We’re using more and more previs to really outline the film we’re going to make—making sure we’re making the right film. Then, we use that to determine how the film should be planned for the practical shoot. There’s a process we use called TechViz, where you're taking previs and mapping it to a space that represents the real-world space the practical shoot will happen in, with objects built to the exact same scale.

You then take that forward to TechViz, where you can inform the film production crew exactly how that practical shoot should play out—how the sets need to be built, what’s going to be in the frame, what doesn’t have to be built, “Is this space large enough to get a technocrane in there?” And it can be as simple as determining what lenses will be used or as complex as using this data to drive motion bases or motion control camera rigs.

Jordan: 31:12

So, does the Unreal Engine stuff ever make it into the final product? Or is it more like another form of previs in a way that-then you're also sometimes saying that movement data gets imported into some other engine?

Chris: 31:23

It has made it into the final product. I think the way to look at it is Unreal Engine is just a real-time rendering engine. It allows you to iterate much faster than offline renderers. Let me talk about PostViz for a second because PostViz is a step where, after the live-action plates have been shot, this will come back to our team, and they will animate on top of those live-action plates to inform what a visual effect is going to look like.

Jordan: 31:53

And I just have to ask, what’s a plate in this context?

Chris: 31:56

So, that would be the live-action frame, the shot—what you’ve actually shot practically on the day. And we would animate on top of that because you can, with that real-time rendering engine, iterate much, much faster than you could if you sent something off to visual effects. Visual effects may take several weeks to several months to arrive at a final shot. And if that's not right, if that’s not correct, you’ve just wasted a ton of money. So, they use a process called PostViz to iterate very fast.

Jordan: 32:28

It’s pre-digital effects, post-shot. Got it.

Chris: 32:33

It is. And it’s all in service of what we were talking about—finding that story and locking it in as quickly as possible using whatever tools are available to you. It’s not that much different from a Ripomatic in the sense that it is getting people to coalesce around an idea. And that could mean getting the director to make a decision on exactly how a creature should move or exactly how an explosion should happen before it goes to visual effects, which is going to create a very detailed, photo-real version of that. That could take several days to several weeks just to render.

Jordan: 33:09

It reminds me, last episode we had a prototyping expert, Bernard Francois, who just does prototypes for games. And we’ve also had Dave Rohrl on, and they both made the point that when you’re prototyping, you don’t want to have the pressure of that becoming the actual game. You don’t want to code it like that’s the game. You don’t want to think about it as the product. And it kind of reminds me of what you’re saying here, where you separate out the final digital visualization with this PostViz, as you call it, so that you don’t have all the pressure of "this is going to be in the real thing" while you’re iterating and making decisions.

Chris: 33:40

The whole purpose is to try different things, and some of them will work, and some of them won’t work. When we do Previs for feature films, we do an average of 2.5 iterations on every single shot. Now, of course, some shots get a lot more iterations, some shots are one and done, but we go in with the idea that we’re going to iterate and try out new ideas.

It’s funny that you say that because at EA, we had a rule that was, you had to throw out the code on a prototype once you were done. And that was very important because you’re doing whatever you’ve got to do to get that to work during that prototyping phase. And then you’ve got to take a step back and ask, "What is the best way to architect this?" Or you could end up with a lot of technical debt down the road.

Jordan: 34:24

Yeah. And engineers, I think from what I’ve understood about EA, they just had a hard time really behaving like they were going to throw it out unless they were being forced to throw it out. When it comes to this world of virtual production, how is that changing things for you and for the industry as a whole? What’s that been like? And I know that change has accelerated recently. What’s been going on?

Chris: 34:48

It’s been good for us, to just cut to the chase. We’ve been big believers in virtual production for many years. Big believers in the concept that you could use real-time game engines to tell better stories for film. Recently, of course, we’ve had a massive shutdown in the industry due to the COVID-19 pandemic. This has created a situation where the adoption of virtual production methodologies has accelerated because it is one of the few ways that Hollywood’s able to get back to work.

I am in my office doing this podcast right now. There are two other people in this building because our workforce is 100% remote these days, for the most part. One of those people is doing a remote motion capture session for a major feature film. That sort of thing, that kind of remote collaboration, was unheard of even nine months ago. It was unheard of that we would have artists working outside of this building, but these days, out of necessity, we are adopting virtual production methodologies as an industry to solve this problem. It’s not unusual for us to get calls now about virtual location scouting, because the concept of putting a bunch of high-value, high-profile individuals on a bus and taking them to tour various sites to scout your locations is ridiculous—we couldn’t even imagine doing that now.

It’s such an insurance nightmare to try to navigate, not only that but a logistical nightmare. You can’t put people together like that, so what you’ll get instead is people who will fly virtual cameras through some tools like Google Earth to be able to identify spots. Then, you could send a very small drone team into capture and put that data through a photogrammetry pipeline and then run it real-time in Unreal Engine, where directors, location managers, and key people on a project can then, in VR, walk around that space.

Jordan: 36:50

That is so crazy.

Chris: 36:51

It’s amazing.

Jordan: 36:52

It's so cool. Can you do them indoors?

Chris: 36:54

Absolutely. Yeah. Same idea. You just fly a drone through it.

Chris: 36:57

Yeah, it can be a drone. It can be somebody with a handheld SLR. You can use any number of tools to accomplish this. And we're doing a lot of LED stage work these days, virtual art department LED stage work.

We did that for The Mandalorian season one, where there were environments built to run in Unreal Engine on this massive LED volume. The concept is that you can get what's called in-camera composites, where you are basically doing your visual effects in real time on stage. With The Mandalorian, you would have had a difficulty shooting that in any other way, because if you were to put a character that is completely shiny onto a green screen stage, the compositing costs alone to remove all of that green would have been cost prohibitive.

Jordan: 37:44

So, I'm not sure our listeners understand—I’m not sure that I understand actually. So, when you say LED, I'm imagining some sort of virtual environment with virtual cameras, but could you explain what you mean by this?

Chris: 37:55

Sure. So what you would get is a volume that is a bunch of LED panels put together, sometimes in a circular or semi-circular fashion, that is acting as a giant TV, effectively. You’ll be running an environment in real time on that "TV" with a motion capture tracking system that sends the positional data of the camera to that engine, so the environment moves based on how your camera moves, creating a sense of parallax.

Jordan: 38:29

Interesting. So is it like a sphere, this thing, or is it maybe a semicircle?

Chris: 38:34

It's a semicircle, typically a semicircle. It could be a flat panel as well. Sometimes it's a box; they're often built bespoke to the production. Though ILM has a Stagecraft offering down here, which is, I think, a 270-degree.

Jordan: 38:49

If you have this 270-degree environment, why does it need to change based on the position of the camera?

Chris: 38:54

If you were to stand in front of an LED screen, so you've got your live-action actor, you've got your real actor on stage in front of an LED screen, if that camera moves, your brain expects to see a movement in the background called parallax.

Jordan: 39:12

Ah, now I understand. The background will move, but it won't move the way it would if it were real, so you need to simulate the real parallax effect.

Chris: 39:20

Yes. Now, the truth is, there are situations where you can accomplish this in a different way. You don't have to have that real-time engine. If you have, let's say, a camera that is going to move in a known way, like on a motion control rig, for example, you could bake out, render out a video that's going to move in the same way. But that's far more difficult and far more constrictive than it is to be able to freely move that camera and have that background react in a way that is very realistic.

Jordan: 39:53

Got it. And this is like, when I hear about things like, "Oh, we made 300 and it was completely made in a room." Is this the kind of techniques we're talking about?

Chris: 40:00

Exactly.

Jordan: 40:01

Got it. All right. And when Halon is doing this for something like The Mandalorian, is this pre-vis or post-vis, or is this like, you're actually making the thing?

Chris: 40:08

This is the final set that will be viewed in the feature. However, it is not unusual for these shots, once they're captured, to go through a visual effects pipeline or a compositing pipeline to sweeten it.

Jordan: 40:22

Right. They're basically the plates, as it were. See that? I'm learning the lingo.

Chris: 40:25

That's right.

Jordan: 40:26

Okay. And being kind of at the cutting edge here with all this technology coming together with these engines and cinema, where do you see things going? Where are you excited about what's going to happen in the next five to ten years?

Chris: 40:39

Well, I think one of the most exciting things I see as virtual production workflows become the norm is that if you're utilizing a real-time engine as the hub for your production, you open up the opportunity to create—gosh, I hate using this word because I feel like it got overused—but true transmedia experiences. Historically, when you make a film, the frame is the final product. What you're trying to get is that picture that is going to be shown on a screen.

This represents, I think, a bit of a paradigm shift where the underlying asset becomes the final product—the digital asset that you're creating. And going back to what we talked about earlier, where you want to tell stories across a number of different verticals, you have the opportunity to then create that asset once. So if you're building environments for a film, those environments can then be used for the game experience, or for a VR experience, or for PR or marketing. Those assets can be used in future iterations of that project. You're creating an asset that has the ability to not only inform different departments on a production but potentially reduce the cost and complexity associated with creating or telling that story on different platforms.

Jordan: 42:07

Makes sense. So, like recently, there was this announcement about how Netflix is going to be doing an Assassin's Creed show. You could imagine that they work out a deal where Ubisoft actually provides the data for these LED sets and so forth. And that helps make the deal happen, even. So we might see content that wouldn't have been able to get made at all. And I'm not saying that's what happened in that deal, I'm just using that as an example.

Chris: 42:31

Yeah, that's right. I'm not sure that happened in that deal either, but I do believe that it's what will happen in the future. You have the opportunity to reduce the cost associated with telling these stories on different platforms, but you also gain the ability to create a much more uniform look.

You're now building worlds that will exist across all of these different platforms. So you won't run into a situation where the game does not look like the film. Or the classic example, of course, is that the images on the back of the box do not represent what you see in the game, right? And I think it's a similar analogy here: you gain the ability to create that asset once. In the film space, it's valuable because you create that asset once, and then you can iterate that asset until you get to that final level of quality. Anywhere along that production pipeline, you can then take those assets for different experiences.

So, an asset built to run real-time for previs or postvis reaches a level of fidelity where it can then be used for marketing, for VR experiences, or other experiences that require a light enough asset to run real-time. Then that asset moves down the production pipeline and gets iterated until it reaches the stage where it's a photorealistic visual effects asset.

Jordan: 43:53

It's an exciting time. And it's awesome that you're in the mix.

Chris: 43:56

It's fun. I really believe that we're on the cusp of seeing a big change in the way we approach storytelling. Even now, just in the film space, the idea that you can do virtual location scouts, capture all of that data, and then replicate that data to run real-time on an LED volume gives filmmakers an opportunity to tell stories that they would have been locked out of telling because it would have been cost-prohibitive to either visit that location or build that set. And that's happening right now.

What I think we'll see in the future is the ability to start looking at stories as persistent worlds that exist across all of these different mediums. I could imagine a situation where you're able to influence the films that you watch, where the lines between what is a game and what is a film start to really blur.

Jordan: 44:55

That's super cool. Chris, do you think there's anything I missed? Anything you want to share with the audience?

Chris: 44:58

The one thing I would say is that this is a really exciting time to be a storyteller. There are so many tools that are available to you. I remember when digital cameras got to the level where you could shoot a film. At one point, it was a big deal. I think House was being shot with Canon 5Ds, and that was a really big deal. But what you got from that era was a lot of storytellers able to tell their stories because it was no longer cost-prohibitive.

What you got out of the explosion of App Store content, what you got out of Steam, was the ability for indie game makers to share their content with the world, where in the past they would have had to go through retail channels. In order to publish a game, it required the use of a game publisher, a partnership with a game publisher to publish a game.

And nowadays, you've got Unreal Engine, that's free to download, you've got marketplace assets. I mean, you can very rapidly learn what you need to learn to start putting a game together. And that's incredibly exciting. That, I think, is going to bring about some very creative content in the future. So I just think it's a very exciting time, both in my space, which tends to be the larger feature film and AAA game space, but also, gosh, it's a great time to be a kid that wants to get into game development. I mean, the only barrier to you doing that is just your motivation to learn.

Jordan: 46:34

I have nothing to add to that. That was beautiful. Let's leave it there. Chris, thanks so much for coming on.

Chris: 46:39

Thanks for having me.

Jordan: 46:42

Thanks for listening to this week's episode of Playmakers, the game industry podcast. In every episode, we have awesome guests like this that bring you real value, helping you produce better games and have a more successful career or business venture in the game industry.

Don't forget to subscribe. Don't forget to write a review. Don't forget to stay in touch. Let's make better games together. That is what this is all about. And if you want to hear all the episodes, all the backlog of amazing guests, amazing interviews, and amazing value, head over to playmakerspodcast.com. That's where it's all at, baby. I'll see you in the next episode of PlayMakers.