The Man in the High Castle: A TV Series with Substance

Lawson Deming (co-founder Barnstorm VFX, Visual Effects Supervisor) – Lawson Deming was on set for the majority of the visual effects sequences and worked closely with the art department on “The Man in the High Castle” Season 2 to help fit the visual effects into the world of the show. He also contributed his modeling, texturing, and lighting experience to the 3D team and introduced the Substance Painter-to-Cycles workflow that was utilized over the course of the season.
Cory Jamieson (co-founder Barnstorm VFX, Visual Effects Producer) – Cory Jamieson oversaw the management of the visual effects team and coordinated the work between Barnstorm VFX and Theory Studios, as well as being involved in the supervision and execution of key effects sequences.
David Andrade (co-founder Theory Studios) – David Andrade managed the team of artists at Theory Studios, which built and shaded many of the CGI assets used in “The Man in the High Castle” and created custom tools to improve the workflow and output of artists. He was involved in everything from asset design and modeling to character animation to the rendering pipeline used on the show.

On Season 2 of “The Man in the High Castle”, Barnstorm Visual Effects was tasked with supervising the creation of roughly 400 effects shots over the course of the season, nearly all of which were completed in-house.
For assistance in creating many of the large setpieces, Barnstorm turned to their partners at Theory Studios, who were responsible for modeling and texturing the multitude of structures in Germania, including the massive Volkshalle (both inside and out). Theory, which employs a distributed network of artists around the world, also designed custom tools to facilitate the secure sharing of assets and to execute cloud rendering on Amazon’s AWS for the large number of complex 3D shots that needed to be created in a short period of time.
Regardless of where they were initiated, cloud renders of CGI elements were automatically pushed to Barnstorm’s server, where compositors incorporated the material into the final shots. Throughout the process, Barnstorm and Theory artists collaborated via Shotgun and Theory’s proprietary MakeTheory software.

Early on in the process, the team decided to utilize a PBR-based shader workflow and made Substance Painter the cornerstone of their texture design. Nearly every 3D object, from smaller elements like cars all the way up to the largest structures, such as the Volkshalle, utilized a combination of Substance Designer procedural materials and Substance Painter texture painting.
Barnstorm and Theory’s 3D pipeline utilized Blender and since it does not natively support the implementation of Substances, a custom uber-shader was created to make it easy to apply Substance materials and render them in Blender’s Cycles rendering engine. With so many assets spread across the workforce, the Substance workflow ensured consistent look and physical behavior among materials designed by different artists.

PBR Workflow with Substance Painter: A Case Study from The Man in the High Castle, Season 2

The original shot included a tall practical sign in multiple languages over a dark night sky that would need to eventually be replaced with a towering CGI Nazi embassy. Additionally, it was requested that the sign be shortened and only include English, which led us to pursue an additional CGI solution. This project would later be important to helping us develop our internal Substance Painter to Blender workflow.

Modeling, UV unwrapping, and initial material assignment take place in Blender. Once prepared, each object that will be receiving a Substance Painter pass is exported to OBJ individually. Single objects with multiple materials also have a UV Color ID map exported as a texture resource to Substance so that it can be used with the Color ID picker mask.

Our project configuration is always set to the Metal Rough preset, which is the set that fits most accurately into Blender’s PBR calculations. The project is also set to OpenGL to further accommodate export into Blender. The model is imported into Substance and texture maps are immediately baked. Custom Color ID maps are also imported and assigned at this time so that material groups can be mapped.

A roughly equivalent metal Smart Material is applied as a base and modified to something more appropriate for the given model. This provides a strong physically-based foundation for the custom painted details in the finishing steps.

Small height, normal, and roughness details are painted by hand onto the base metal material, allowing the following procedural and particle effects that simulate the paint and weather to be affected accurately.

A fill layer with adjusted roughness, metallic, height, and normal values is applied over the metal base to simulate paint. This is set to a lower opacity to allow some of the base’s worn metal look to come through.

Localized grunge is applied using a combination of custom paint strokes and color/roughness adjustments driven by generator masks and curvature/AO maps. When painting custom grunge or damage, we take the full context of the shot into account – which areas are the most exposed to the elements, weather directionality, where they might be touched or brushed the most often by pedestrians, and similar factors. Authentic details like that are critical to selling a photoreal element, especially when it’s the focal point of a shot.

Paint bumps and chips are created using custom brush strokes on a height-only layer. The effect can be further modified to create general surface displacement by adding a blur filter to the strokes or generator mask, which makes the model feel more hand-crafted and real.

Overly dark edges where the AO map has masked in grunge too aggressively are lifted using a partially-transparent color and low roughness adjustment driven by a generator edge mask.

Leak and Rain particle effects are added over the top of the pole material to give it the feeling of being outside in the elements. This adds a final touch of environmental authenticity to the material before exporting the maps.

The PBR Metal Rough map set is exported in 2K or 4K from Substance and imported into Blender, where they are plugged into our internally-developed Substance node. This shading process is repeated on all foreground, middleground, and background assets until the scene is completed and interacting accurately.

In some instances, here with the sign itself, we use image textures created in other software in conjunction with Substance maps mixed into our custom PBR node. Here, the image of the sign (left) has been mixed with the Metal Rough set export from Substance (right) to build the material on top of the sign image. The material in Substance was built in black and white with this usage specifically in mind:

Last, a final custom greyscale displacement map (created in Nuke or After Effects) is mixed over the top of the Substance displacement maps to create indentations for the bolt models in addition to the Height maps.

This wireframe, taken from Blender, shows the complete extent of the shot replacement done in CGI.

The final shot, rendered in Cycles and composited in Nuke.
Here are some comparison shots between early blockout and final scenes textured with Substance

A big thanks to both Barnstorm VFX and Theory Studios for this very detailed user story!