How is virtual production revolutionizing the film industry?

In Take 5, Sheridan's thought leaders share their expert insight on a timely and topical issue. Learn from some of our innovative leaders and change agents as they reflect on questions that are top-of-mind for the Sheridan community. 

In this installment, Spencer Idenouye, Lead of the Virtual Production Innovation Hub at Sheridan’s Screen Industries Research and Training Centre, speaks about the application and potential of virtual production. 

1. What is virtual production (VP) and what are some of its applications within mainstream media? 

Virtual production uses different tools to create an immersive, visceral environment that is an accurate, live representation of a set. Tools like LED walls and large-format displays can render digital environments that actors and crew can see, allowing everyone involved in the production to get on the same page and understand what the scene looks like in real-time. VP can accurately capture lighting and reflections and reduce unwanted spill (unwanted remnants of the green screen that are challenging to replace), which can be difficult to fix in post-production when only using a green screen. 

Virtual production can also encompass the use of a green screen in combination with a game engine to render images on the screen in real-time or involve performance/motion capture where an actor is puppeteering a character. 

In terms of mainstream application of VP, think Disney’s Mandalorian or HBO’s Westworld. Both use VP to create complex digital environments. A more functional, practical application of VP to solve a logistical problem would be shooting the interior of moving vehicles, which is something we’ve done at Sheridan’s Screen Industries Research and Training (SIRT) Centre. In place of specialized rigging, cameras on a vehicle, blocking off a road or building a fake streetscape on a backlot, crews can have this type of control with the use of a digital background, and not actually have to drive the vehicle at all.  

2. How does VP offer potential solutions for an industry impacted by the COVID-19 pandemic? 

VP has enabled opportunities for continuing with production where travel is restricted. If you can’t send a crew to Paris to film, the cityscape can be simulated at a local studio with a VP set-up.  

During the pandemic, VP also allows for collaboration with crews around the world. Game engines, like Unreal Engine, have developed toolsets for more efficient collaboration that allow for multi-user sessions from various locations. SIRT uses Unreal Engine to facilitate these kinds of interactions, as well as other hardware and software tools, to fuel powerful and efficient remote working sessions.  

3. In what ways has VP revolutionized the screen industries?  

In addition to the global collaborative potential, VP may also save money and time for a production. While SIRT isn’t in the business of production scope, there’s a case to be made for at least the reallocation of budget from location costs to VP. There’s a lot of money spent and perhaps wasted on transportation and going on location to shoot. There are permits involved, and it’s not always easy to predict the length of a shoot or if re-shoots will be needed. There’s potential to save money by creating these same environments on a soundstage.  

Where I see VP having a huge impact and being embraced across genres is in the creative development process. Using VP can be incredibly collaborative and visceral. Rather than storyboarding or doing computer visualizations, you can spend time preconceiving and put it up on one of the LED walls. You can have a play session to figure out the ideal location and set-up. Normally this could be done on location during rehearsals but with VP, you could do it virtually in advance.   

4. What are some of the limitations with VP technology? 

It’s important to think of VP as a toolset and not a solution to replace everything. It needs to be leveraged in the right way and for the right job. Its use is mostly seen in sci-fi TV shows and films or effects-driven productions because it’s hard to create those environments on set.  

There is also a question of access to the technology and cost. It doesn’t have to be expensive as there may be parts of a production’s budget that are reallocated to VP and create efficiencies in workflow.  

Just to give you a sense of our set-up at SIRT, we have what we call at 10,000-sq.-ft. smart stage with a 3,000-sq.-ft. truss system, 40 optical motion capture cameras, large format displays, digital cinema cameras, and high-end computer workstations. That’s only on the VP side of the house. Not every production company has access to a set-up like this – there is an investment required.  

5. How is SIRT helping forge a path in VP? What’s on the horizon for the Sheridan research centre in this space? 

SIRT’s focus is on solving workflow problems and fostering innovation. We’re launching a Virtual Production Innovation Hub (VPIH) that will allow us to be seen in the industry as the place to do VP applied research and give us the space to train more students in VP and collaborate with industry partners.

In terms of forging a path, we’re working with any Ontario (or Canadian) company that’s new to VP to figure out how we can help integrate these tools into their production pipelines. SIRT actively reaches out to companies to educate them about VP, how it might benefit their work, discuss research opportunities and identify potential funding avenues. 

VP is an emerging area and will advance in ways that we can’t predict. We’re ready, through our VPIH, to move forward with experimental endeavours, take risks and figure out how to work with hardware and software to push the creative boundaries in ways that will positively impact the screen industries.  

Spencer Idenouye is the Virtual Production Lead at SIRT. As part of a collaborative team, he’s creating an integrated virtual production studio for training, applied research and commercial services. He provides leadership and expertise in applied research and technical research for the movie, television and gaming industries, and is focused on the application of VR technologies. Spencer’s education includes a BFA in Media Arts & Digital Technologies and two post-graduate diplomas in Directing and Editing in Advanced Television and Film and Computer Animation Digital Visual Effects. His expertise includes directing, editing, VFX artistry, cinematography, digital content production and motion capture. 

Interested in connecting with Spencer Idenouye or another Sheridan expert? Please email communications@sheridancollege.ca.  

The interview has been edited for length and clarity.