This talk was first presented at Siggraph 2019 by MPC R&D London Head of Production Technology Kai Wolter.

Authors

Kai Wolter(MPC R&D)
Luca Bonatti(MPC Film)
Marco Rolandi(MPC Film)
Julien Bolbach(MPC Film)

Abstract

The Lion King presented the unique challenge of creating a full CG feature film that could cross the border of photo-realism and be perceived as live action by the audience. The director and the production design team strongly pushed for a naturalistic look, heavily influenced by the imagery of African landscapes made popular by documentaries. The challenge that the MPC Environments Team had to face was to create a world that could be fully explored by a wide variety of virtual lenses, including dramatic wides and tight telephotos, used to frame the action in the vast spaces of the savannah.

Context

After an initial stage of photographic research on the real locations that inspired the movie, the first step was to build the 150 square kilometers of terrain that extends from Pride Rock – home of the lion Simba – to the massive spaces of the Pride Lands, traversing all the different ecosystems that the story touches. These huge sets featured elements of real African landscapes as well as the many bespoke landmarks that define the world of The Lion King like Pride Rock, Elephant Graveyard, Cloud Forest (etc.) for a total of 66 unique smaller – but still massive – sets. In the ambitious attempt to give complete freedom to the director, a simplified version of the sets were used directly in Virtual Production, allowing the entire movie
to be shot live in VR. Given that the characters and the cameras were immersed in the vast African landscapes since the very beginning of production, it soon became clear that the entire world had to be recreated in 3D at full resolution. When working on The Jungle Book (2016), the Environments team heavily relied on Digital Matte Paintings to extend the 3D sets, which were mostly featured in the foreground. With The Lion King, the VFX Supervisors challenged the team to abandon the use of 2D elements and hence deliver the totality of the digital sets to the Lighting department for rendering. The clear benefit was to be able to render the entire shot in Pixar RenderMan and to deliver to the artists in Compositing not only images that were very close to the final look, but also the necessary amount of AOVs and deep data. The use of very long lenses was an important part of the cinematography of the film and the 3D data
produced by the render engine allowed control and photorealism when adding depth of field in compositing.

The existing pipeline was expanded to make space for a new procedural technique for the creation of high-resolution natural elements (such as rocks and terrains) and for a dynamic approach
to set dressing. Bespoke UV texturing of the individual assets has been almost completely replaced by a more organic approach that considers the set as a whole. Using custom, complete sets of texture maps scanned on real locations, the Lookdev team built shaders that procedurally applied textures and materials replicating natural phenomena such as weathering and erosion, always keeping the maximum resolution of the original maps and avoiding any kind of baking in UV space. This allowed to build photorealistic shaders that
could give the desired look in the wide camera angles as well as in the many macro shots of the movie. A brand new framework named PaX was specifically built to process the huge amount of assets present in the digital Sets at build level and efficiently optimize it for every shot, streamlining also the contribution of the different disciplines of the Environments team (set dressing, environment dynamics, effects).

Sets Analysis and Creation

The Environments team carefully studied reference images of the African natural environments and produced a collection of ecosystems.
Each one of them was defined by very specific features: the collection of assets belonging to a well defined geographic area (rocks, plants, trees, etc.), the set of rules which mimic their natural distribution on the terrain and the procedural geometries, shaders and textures which would go with it. While on other previous
projects most of the authoring work on the sets was done manually and each iteration of the process was baked into static assets
[Cieri et al. 2016], with the new approach, which was structured in modules, each module was not baked data anymore but the recipe for the creation of its contents. A wide set of tools for Houdini developed within the Environments department allowed the artists to load the pre-built ecosystems and automatically generate a believable natural scenery that could take into consideration not only the relationship between natural elements, but also the creative
needs of the artists working on the shots. The team was able to procedurally increase the detail of soil and rocks, distribute debris mimicking the effects of erosion, grow vegetation in relationship to terrain’s features such as rivers, mountains, etc. This approach also allowed artists to develop exponential layers of complexity by using the output of one iteration as an input for the next. For example we generated vines which grow on the branches of intricate, procedurally generated forests of trees. The inclusion of procedurally generated geometry was a significant part of the process, and allowed the creation and placement of individual assets which were blended together at a later stage in a cohesive set. Typical examples were the Canyon and the Pride Rock interiors sets, in which sand
was added in interstices and gaps between assets in order to have seamless surfaces, flexibility and natural looks. In the Pride Lands set, hundreds of kilometers of savannah plains have been scattered with billions of instanced grass blades and acacia trees that reach the horizon. Such a dynamic approach gave the artists the freedom to modify the main landmarks (from trees to mountains) and see the set dressing automatically re-adjust itself around them.

Pax

The huge amount of data produced by set dressers couldn’t be passed to the render engine without an efficient framework for optimization, especially considering the memory constraints dedicated to the environments for each rendered frame. Sets were ingested in MPC’s pipeline with a similar technology to the one used by the FX department. Every subset of scattered elements was in fact stored as an InstancePkg, an MPC-specific package containing one or more particle systems that describe the placement and instancing of the assets present in the project’s database.
PaX (Particle Xccelerator – which also means “Peace” in Latin) was introduced on The Lion King as a framework to filter the data created at a build level and dispatch it to the 1478 shots of the movie. It was a very close collaboration between the R&D team and the departments involved. Rather than hardcoding the behaviour of the tool through lines of code, the process to convert one environment in a shot context was handled by a customisable template
that allowed the different teams involved to contribute and add value to the shots. PaX was used first to optimise each shot by pruning unnecessary instances through frustrum and occlusion culling techniques, also assigning LODs to each instance based on its size on screen. By using PaX, environments were in some case
up to 99% lighter than before, reducing memory consumption and render times drastically. Its flexibility, by design, quickly allowed for more specific needs. The Techanim department was able to use PaX to add some complexity to the shots, by swapping static models with pre-cached animated models to add wind to the shots,
with shot-specific settings when required (wind direction, strength, etc.).
They could also promote instances that would require more accurate simulations and feed it back to our usual hero dynamics simulation pipeline. By storing templates per sequence or shot, PaX could trigger automatic updates, for example when the camera or the animation of the characters changed. Throughout the project, PaX allowed the teams to add further customisations, like splitting the instances by distance from camera to facilitate 2D depth of field
or merging by instance type to take advantage of more efficient instancing. The seamless integration of PaX with MPC’s pipeline allowed a streamlined workflow and an efficient management of large scale sceneries, allowing the Environments team to efficiently render incredibly complex and vast landscapes within production’s memory and time constraints.

Acknowledgements

Special thanks go to Audrey Ferrara, Marco Genovesi, Ignacio Urruti, Jules Bodenstein, Marc Menneglier, He Sun, Filippo Preti.

References

Stefano Cieri, Adriano Muraca, Alexander Schwank, Filippo Preti, and Tony Micilotta.
2016. The Jungle Book: Art-directing Procedural Scatters in Rich Environments. In Proceedings of the 2016 Symposium on Digital Production (DigiPro ’16). ACM, New
York, NY, USA, 57–59. https://doi.org/10.1145/2947688.2947692

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
SIGGRAPH ’19 Talks, July 28 – August 1, 2019, Los Angeles, CA, United States
©2019 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5820-0/18/08.
https://doi.org/10.1145/3214745.3214753