The first row in many rooms is now reserved for lecturers so I had to abandon one of my favorite seating places of past years. Further the exit of the main auditorium has been moved to its side, probably because people pushing their way out and people pushing their way in made the access to popular lectures a pain — literally. Further, the FMX had matured and grown and hence offered even more parallel lectures in the opposing building.

Shortly before ten I had my ticket, had set off my first tweet, got my welcome bag, had my ballpen at hand and took a seat in the second row: I was ready for this year’s conference! Since I was a bit early I came to see the last of the screened animations of the Filmakademie Baden-Württemberg, a satirical take on the juicy topic of Austrofashism in the 1930′s.

Going Mobile

Neil Trevett,
originally uploaded by Phil Strahl.

Then they rolled this year’s FMX trailer, in my opinion a fresh idea after last year’s rather featureless animation. Then the first speaker of the event got introduced, Neil Trevett from nVidia, incidentally nVidia being the event’s main sponsor. I was preparing myself for having some marketing mumbo-jumbo dropped onto me but it wasn’t that bad after all in his lecture Movie Making and More, All in the Palm of Your Hand.

First he lined out how proud nVidia is of their Quadro series, CUDA and that all of this year’s Oscar nominated VFX films used nVidia technology somewhere in the course of the production. Big whoop — Adobe can claim the same thing probably.

Then followed some more nVidia-green Power Point slides depicting how the market of the “classic”, stationary desktop computer was more or less saturated today, even the growth for notebooks and netbooks started to level out, yet the market for mobile devices such as smartphones and tablet-computers exploded exponentially, with the Android operating system already ahead of Apple’s iOS. And since nVidia wants to get a big slice of that cake as well they started development of the Tegra series, high-performance graphics processors with as little power consumption as possible.

But that’s not enough: The old x86 chip architecture is also in decline since tablets and smartphones don’t come with a constant connection to a power grid, so the old ARM architecture will become a big thing of the near future. For nVidia this is Project Denver, ARM with integrated GPUs, while Microsoft’s Windows 8 will also support this kind of hardware. nVidia also works together with Adobe to make the execution of Flash and AIR applications more economic on mobile devices. “Apart from playing back video, the main task of Flash seems to be blending and filtering. And we let the GPU do this.”

nVidia seems to be pretty tight with Motorola as well, because their cutting-edge processors chips are and will be inside cutting-edge Motorola hardware as a chart showed. And I realized that I will carry a laptop with me for a couple of years longer, because what I want and need to do on the go can’t be feasibly done on a tablet anytime soon. But for the average person who wants to listen to music, surf the web, read eBooks and write emails a tablet with a full-sized keyboard/dockingstation/battery-combo (like the ASUS Transformer) might be just what they need. USB-input-device support anyone? Ice Cream Sandwich?

In the very near future mobile devices will have quad-core processors hence multitasking should become as common as necessary, further the portable devices will provide autostereoscopic displays, such as the Nintendo 3DS. Last but not least the resolution with increase drastically to an estimated 2560 by 1600 pixels (nVidia boasts it “Extreme HD”) which effectively results in a resolution of 300 dpi on a 10-inch tablet — print quality.

But what about power consumption? “As paradox as it may sound but more cores save you more power: The excessive cores only turn on when an application needs them” Neil explained.

Gee thanks, Neil, that’s pretty interesting, consumer-wise, but what about our industry? “Tablets in the movie industry can be and currently are used in three different ways,” he continued. One possibility is to port software to mobile devices. I snorted mentally: Anybody who tried the incredibly limited version of Photoshop for Android knows that a smartphone is still way too underpowered to get real work done. “By 2014 mobile devices will have 100 times the performance of today.” So will CUDA also work on phones? “No, CUDA’s power consumption is one of the biggest stepping stones in that respect” Neil shrugged. But that didn’t matter to me, because “the next generation of tablets and smartphones will be aimed at the demands of artists and the creative, like by additionally offering stylus operability and pressure sensitivity. The first devices will probably be released this year.” I dumbly smiled at the poor person sitting to my right. I am so buying into this!

The next useful integration of tablets was wireless tethering with desktop applications, because “this ‘new’ way of interaction with a user interface that is so much more intuitive than having to nudge a mouse pointer across the screen.” Adobe released for developers the Adobe Photoshop Touch SKD which opens developers the door to think of clever ways to integrate a tethered device. For example, it could serve as a color palette where swiping blends colors together. The resulting colors then can be returned to the host application. EDIT: Adobe recently me an announcement via eMail to update Photoshop and get complement software from the AppStore. Pity I roll with Android.

The third possibility is using tablets as cloud-clients. nVidia has already a Flash-based technology for combining collaboration with Tegra-powered tablets called “Studiopass”: It lets you upload, stream, annotate and comment in real time on video files with others. It is also possible to use the tablet as intuitive virtual viewfinder that’s connected to a render farm that returns within a few seconds the rendered image to the device. And, surprise, Studiopass is built on Flash.

Just when Neil got glowing eyes and popped up a slide in purest nVidia’s corporat0- design reading “Super” Computing, the time was up. Close call!

Baking Light

François takes a pic,
originally uploaded by Phil Strahl.

I was too lazy to cross the street for Approaching a CG Production so I stayed in the König-Karl-Halle for Bakery Relight by Thomas Driemeyer, Emmanuel Turquin and François Belot from The Bakery who talked about their company and their software, Relight. I was a bit distracted by the amount of people swiping away on their iPhones, Joseph Olin among them, while François outlined the history of the company in southern France. I only got to hear the last few words, “Sand, Beaches and Girls!”, although what I understood as “Beaches” could have been some other word. Or maybe it just was the thick French accent and the tendency to pronounce some words French, or switch to French vocabularies entirely.

Relight is essentially a tool for lighting and shading shots and to develop looks, realistic as well as artistic. “This tool was designed by artists and not by engineers” they noted and indeed: As Emmanuel showed a quick overview of the interface it seemed quite straightforward, a bit like Katana. “You don’t need to make a break every time you want to render a change on your way to the final shot, you always work with the final and that really fast.” Even little comfort features are implemented, like solo-ing certain light sources.

It plugs directly into Maya, 3dsmax and XSI, it comes with its own hair and fur system that is created at render-time by its production renderer that produces really good and clean images. The renderer itself is a Reyes-like rendering algorithm, iterative and optimized for fast feedback in production quality. But this need for heavy-duty-caching takes its toll on memory consumption I guess.

Once the scene’s visible geometry is cached by renderer (many millions of polygons take a couple of minutes) Relight unfolds its performance. Lighting and rendering a forest-scene with hundreds of trees and atmospherics only took a minute on an average notebook computer — impressive. And once the geometry and the first lighting pass is cached, the renderer gets even faster with every change. It keeps track of what has been touched or adjusted and only updates the dependencies, what had been affected by the change; and nothing more. “Depending on how many processors you use, the suite scales pretty well accordingly.” Even motion-blur and DOF (depth of field) are fast.

Another big thing of Relight are point clouds for quickly carrying out calculations that would bring a raytracer too its knees, memory wise. Point clouds (or disk-clouds to be more precise) are independent from the underlying geometry and make it possible, for example, to render ambient occlusion in fur stunningly fast. But ambient occlusion is only one application. Point clouds can be used for glossy reflections, environment light (from HDRIs, for example), area-lights or sub-surface-scattering.

Towards the end, François added that since Relight provides feedback so fast, it is a great tool for budding lighting artists and there are many collaborations with schools. Then he closed with a sentence in thick French accent that sounded to me like “Thank you very much for your invention!”. You’re welcome!

Arnold, the Tracer

Whereas The Bakery‘s Relight tries as much as Pixar’s RenderMan to cheat around raytracing, Sony Pictures Imageworks has a completely opposite approach with their proprietary renderer Arnold. The lecture by Marcos Fajandro and Larry Gritz Path Tracing and Unbiased Rendering started off by comparing rasterizer/Reyes renderers and ray-tracing.

Of course, Reyes depends on shadow maps and offers no ray-tracing but can be insanely fast, whereas a pure ray-tracer craps out completely with too complex geometry (see the issues with Davy Jones as discussed on the FMX/08) and/or/like hair.

So what do you do? Integrate some limited raytracing into a Reyes renderer or hack rasterizing & (deep)shadow functionality into a raytracer? Both options aren’t exactly irresistible. So the folks at the Spanish company Solid Angle, I haven’t heard of ever before, developed Arnold.

The first feature film rendered with Arnold was the, in my opinion terrible, Monster House and now since a couple of years it has become Sony Pictures Imagework’s only renderer.

Arnold is based physically on the Monte-Carlo methods, it uses no storage for lighting and global illumination information, has only one quality knob to tweak on (namely the sampling) and outputs the final in a single pass. Yes, Arnold is a ray-tracer par excellence and traces millions of rays. “And when you trace so many rays you need to make the rays really fast”. Further, Arnold sports networked, programmable shaders, subdivision surfaces and yet can handle “hundreds of millions of triangles and hair splines” and virtually “hundreds of gigabytes of texture-maps”. Impressive!

Since everything (everything!) is ray-traced, the final image is optically seamless. There’s no need for say, a shadow pass or motion vectors. My worst fear of such a renderer is that it probably takes ages just to get a feedback of lighting changes, but that is not the case: An example video showed a car in Maya in the viewport whose camera was turned around it. As soon as the operator let it go, Arnold kicked it and its interactive mode rendered the selected region in big blocks, then smaller blocks and even smaller blocks until you could make out the raytraced details of car paint and reflections. This refinement process took not more that two seconds.

“In the end what matters is not how long you wait for your beauty-rendering but how long you wait for feedback of each iteration in the (re-)lighting of a shot. Since an artists costs you approximately $ 40 an hour, this way of saving time saves you a big amount of money and the artist downtime between iterations” Marcos explained. I heard several mental kaching-sounds in the audience.

Making raytraced motion-blur efficient is also important for the final rendering. A fully lit and shaded scene with production-quality deformation motion-blur rendered only 15% longer than without. Arnold even is proficient when it comes to volume renderings.

“The joy of Arnold is, that everything just clicks together seamlessly when you raytrace everything in your image. If you use different renderes for different tasks, even simple things like shadows can be a problem, like shadows in volumes. With Arnold there’s no catch — you get the whole package in one go with ambient occlusion, motion blur, soft shadows on motion blur, caustics, etc. ” Marcos summed up.

And just as joyful is instancing of geometry that gets loaded into the memory once for whatever number of instances. One shot of Alice in Wonderland had 61 million triangles and could be rendered in a single thread’s 15 hrs which boils down to an hour or less in a farm.

“Still there is lots of room for future improvements” Marcos concluded before Larry took over. Larry talked in more detail about Sony’s way of VFX and relighting using the upcoming Smurfs movie as an example.

For every shot with CG-integration, they shot a HDRI of the set with an incredibly expensive Spheron camera. In fact, not just one panorama, but two in different heights so with the set survey data it is possible to calculate and to recreate the scene geometry via triangulation roughly in CG. Then, for quality reasons, the light sources and instruments in the panorama get replaced by Arnold lights and erased from the HDRI panorama via Katana. The actual plate from the camera is projected onto the low-poly scene geometry, then the HDRI panoramas get also projected into the scene. “It’s kinda the same what was done for Benjamin Button” Larry added. Now the CG-characters are imported into the scene and get their lighting and bounces from the surfaces that the plate and panorama was projected onto.

As mentioned above, Arnold has an open shading language. “Traditionally, shaders are black boxes to the renderer that just return some values to it; their units are sloppy oftentimes and their underlying C/C++ code can also crash the renderer altogether. With an open shading language, we don’t have these issues.” Instead of returning color values to the renderer, the Arnold shaders compute so called “closures”, descriptions of how the surface will react to light, and pass these numbers on to the renderer, which can decide what to do with them. “This is also 20% faster!” Larry added happily.

What working with these “closures” exactly meant and what it had to do with ray-budgeting and Multi-Importance-Sampling (MIS) for the Monte-Carlo raytracing I did not fully grasp but Larry had some links (and his email address) on the subject, in case anyone was interested. I reproduce them here for the same purpose:

• http://bit.ly/s10shaders
• http://opensource.imageworks.com
• lg AT imageworks.com

Some features of Arnold like sub-surface-scattering are performed with point clouds on-the-fly, although the renderer still employs Monte-Carlo methods when point clouds won’t suffice. Since Arnold is proprietary, Imageworks developed a plug-in for XSI themselves and a very basic one for Maya.

But why did Sony Pictures Imageworks settle for just Arnold a few years back? “The pass management grew so complex and bloated, it was hacks upon hacks, that overwhelmed the mental capacity of the TDs. Now it’s back to just having a single pass. Lighting got so much faster and in the end. Lighting and rendering a single pass still is faster than rendering lots of different passes and trying to get them working together.” Larry explained, and Marcos added: “When you reduce time somewhere in the process, however, somewhere somebody uses the new freedom to add more complexity.”

Currently Arnold uses only the CPU despite the trend of making everything CUDA- and hence GPU-compatible. “Porting would be complex and time consuming and in the end wouldn’t speed up things significantly, so we spend the efforts to optimize the CPU code instead. Further, the terabytes of textures would need to be streamed to the GPUs as well…” Larry summed up the bottleneck situation.

“We still have a couple of minutes left so I’m gonna show you the Green Lantern trailer. Everything was rendered with Arnold.”

Fun fact: The plate at the end of the trailer with the title and release date also had a rather small line reading “Also playing in 2D theaters.” Like it or not: Stereo has finally arrived and it is here to stay. Deal with it!

And here’s a LaTeX-set presentation on Importance Sampling for Monte Carlo Ray Tracing from 2006 with lots of pretty pictures (and some equations).

Coffee!

Haus der Wirtschaft,
originally uploaded by Phil Strahl.

The few hours I was asleep took their toll and I was wasted for the break. I schlepped my bag and camera to a long queue in front of Starbucks where I got me a motivational Caramel Macchiato so I had the energy to look for food. I found an Asian noodles stand in a subway station and made my way back to the Haus der Wirtschaft, munching on fresh cooked vegetables. The remaining ten minutes I had an espresso in the showroom of the FMX. When I felt ready for some more lectures and headed into the König-Karl-Halle and was surprised to see all the good seats taken. So I settled for a suboptimal seat next to a German student who was constantly eating or drinking something and merrily ignoring the ban on recording devices. Funny thing though was that he thought I didn’t speak German and I was in no mood to shatter his belief. So I sat there with Ophelia, my notebook, in her bag on my lap and waited eagerly for the next lecture to take place.

Render de Janeiro

Building, Lighting & Rendering “Rio” by Blue Sky’s Andrew Beddini was up next. Andrew somehow reminded me of a friendly Ben Stiller character and his engaging lecture was a pleasure to listen to without once the need to close my eyes for just a couple of hours. Or maybe it was because I was so close sitting to the pumped-up loudspeakers that I feared my ears would pop as they rolled the FMX-trailer.

Blue Sky, founded in 1987, is a veteran of the industry, they even took part in creating CG sequences in Disney’s original TRON. The next milestone was the animated short Bunny in 1998 I remember seeing at the Ars Electronica that time. I even remember that it was the first animation to employ the nowadays obscurely named Radiosity.

People really like to talk about their renderers I realized, as Andrew started summarizing the features of their proprietary renderer with the featureless name CGI Studio, or just Studio. On a rendering from 1993 Andy demonstrated the features their raytracer could produce back then which they still use today. Some of its features used for Rio were secondary rays, true radiosity, raytraced soft shadows, tesselation of beziér patches, a procedural shading pipeline and the possibility to use procedures for much more than just shaders. Rio is a feature of 1800 shots, and 80% of it had to be done in just 5 months so the focus was on creating a pipeline that was fast and efficient.

Character lighting was one of the main and foremost concerns of the production. And “good lighting needs good development.” The character of Susan with fair skin, glasses and big eyes was the first testing ground for plausible shading and lighting. First they applied the sub-surface-scattering-shader to her skin and realized, oh boy, that it looked not really convincing. So the first thing was to implement a density adjustment into it, then light transmittance through the skin, radiosity and a subtle but necessary secondary transmittance model that made the skin softer still.

Eyes behind glasses were another challenge because “eyes are critical for emotion. You really want to get the eyes right first. If they are off, if the audience doesn’t buy it, then all the other efforts are in vain, so get the eyes right!”. First they raytraced the eyes and the glasses but the physically realistic look did not work with the stylized character design. For example, the shadowing of the glasses under the rim was just too dark with the realistic refractive index of 1.5, only 1.1 was just about right. Can you do this? Should you do this? Andrew made it heard: “Break reality to make it work for you!” What was behind the glasses was rendered separately with a plate of the background to yield realistic refractions of it, as for the reflections, the reverse angle shot was used ever so subtly. But still the eyes didn’t look alive, so Blue Sky went the whole nine yards and rendered for the eyes a UV pass, a reflection pass, an object pass so the highlight could be adjusted in Nuke accordingly. This finally gave the eyes that certain something that was missing and was tweakable in every shot.

The set creation of the favelas was a difficult task as well. After the first color studies the assets were created with low complexity in a modular fashion and could be combined to a very organic whole according to a plan of the set’s final layout. “If you want to sell a set to your director render it in gray with just the ambient occlusion and he’ll love it!” Andrew joked. “There were absolutely no texture maps involved, everything was done procedurally with using the world space coordinates. Also we don’t need a level-of-detail (LOD) system.”

With just one building block, the lighting department instanced the geometry for lighting and mood tests of what the location might look in broad daylight, in the afternoon, at night and with atmospheric effects.

Raytracing and motion-blur — another dreaded combination of mine. But to tackle that problem Blue Sky employed a lot of tricks that made their lives easier. One of those was modulating the render-resolution accordingly via script: “At some point you can’t tell whether a 2k image was blurred or a 1k image was blurred, so if the renderer dials down the resolution you effectively are four times faster.” Another trick was using Nuke’s excellent vector blur: The camera and a Z-Depth channel from Maya got imported into Nuke and was blurred there in post.

A slide depicting the favela at night appeard on the screen. “In this particular scene we had about ten thousand light sources” Andy spoke and paused for emphasis. Ten thousand light sources with raytraced shadows?! “The thing is,” he continued “we use the same pool for all shadows, they get calculated at the same time. So if we’re having one light source or ten thousand only adds 10 to 15 % to the overall render time.”

Since the protagonists in Rio are birds, the sky is an important part of the film as well. “About 30% of the picture is sky!”. So how do you get a beautiful art-directed sky? The traditional answer is to have talented matte-painters. “The problem is, that we had four to five matte painters but hundreds of skies to paint. So this wasn’t feasible. Further the feature is in stereo, so…” he switched to the next slide depicting various clouds on a black background “…we had our art director paint a style guide for clouds we then built in 3D with volumes. We rendered each cloud with surface normals from 38 different camera tilts so you had almost any angle. Then we imported those onto planes in Nuke and populated the sky in the 3d-space there with the clouds.” The normals made it possible to vertex-light the cloud planes in Nuke. “This is really fast and you can churn out skies quickly. We did this for an entire sequence at once, not only for a single shot.”
But sometimes you need hero clouds, especially when they needed to have a volume and depth or some advanced lighting effects, like transmission on the edges of a key light. These clouds got individually modeled and rendered. The atmosphere-gradient with the sun was also done as a dome in Nuke and when you throw everything together — voilá — there’s your final sky! This sky then got rendered in Nuke and was used on a plane in Maya for the reflections on the water.

Generating a vista of Rio de Janeiro, especially a stylized version that still looks as photoreal as possible was the next challenge on Rio. The foundation for the environment was survey data of the topography that got artistically adjusted with some liberties. Still, it was built to scale so the renderer would deliver correct results of atmospheric effects, for example.

The vegetation was, like almost everything in Rio, procedural with implicit surfaces. But here the procedural approach posed some difficulties: it is hard getting a procedure to a single point. As for the shading, two world space procedural textures were created, one for the granite and one for the plants. A simple rule that depends on the steepness of the topology then blended between the two, so vertical walls would have no vegetation and soft slopes would be fully covered in trees. Finally, the city’s buildings were roughly modeled, but also were designed for procedurally driven variety: They could have differently spaced windows, doors, floor-segmentation, surfaces and rooftop structures.

What surprised me the most was the way the lighting was defined and carried out: It was script-driven and those scripts were simple text-files the production edited with NEdit in Perl and Python. “This is great because the files are small and portable, you can send them securely via email, and you don’t need to open Maya just to twist a light a bit to one side.” Blue Sky even has an interactive front-end called “Quick Render” to render models and light situations without waiting for Maya to finally launch. And there was more: “Light sources are not defined in the world-space but in camera-space,” instead of XYZ they are described by values for rotation, elevation and distance from the camera. Again, you don’t need Maya to adjust a light; creating a consistent rim-light can be done with a few lines in a text-editor easily.

Finally, even the atmospherics were rendered in favor of a Z-pass. “A Z-pass always has artifacts along the edges so you need to render it really big.” And once you have time for that and to add fog & haze in post, you can render physically much more plausible (and within the scene reflected) atmospherics.

A Hairy Subject

We only had a few minutes to let sink in what was heard before Mohit Kallianpur from Disney Animation Studios continued with his lecture of Untangling “Tangled” where he was the Look & Lighting supervisor and described the history of the look of the movie.

In 2007 the movie had a much darker, browner tone to its concept art until John Lasseter intervened and pointed the production in a more colorful and saturated direction. Still, that was relatively late in the pre-production stage and there was not that much time left to get the movie done. So Mohit got down to the root of art direction and formulated three principles: Stylized shapes, illustrative colors and believable textures.

Shapes

Then the research begun by watching and analyzing the shapes, colors and appeal of the old Disney classics Snowhite, Pinocchio and Cinderella. Especially the latter had a certain shape language of flowing curves visible in almost every shot, a graceful harmony. Moreover, a set of signature shapes of that movie was collected consisting of various bell-shapes, s-curves and leafs. Everything in the film would follow these shapes, even the canopy of the trees.

Based on a very impressionist and rough mood-painting that followed the shape-guide, the team produced a full CG version of it as reference. This made it obvious that the language of shapes worked well, but the painterly appeal of the surfaces was too stylized. So the world needed to have believable textures, not rely on impressionist suggestions of detail.

Architecture

The architecture of Tangled was influenced by European cities, Disneyland (!) and Pinocchio: Everything should be small, friendly and approachable. The buildings are not tall and they flare out in a curve, they appear even chunky and beefy with no sharp corners and a very organic and hand-built feel. And like this Mohit wanted the architecture on Tangled to look: old and used but not decrepit or dirty.

Color

But what does “illustrative color” now mean exactly? The production settled for a lush saturated palette and always a play of warm against cool: If the light is cool, then the shadows should be warm and vice versa.

Legally Blonde

Yet the most intricate and most critical task was to get the look of Rapunzel’s hair perfect. Much like the sky in Rio, the hair in Tangled was a character by itself. The first tests showed that the hair shaders the studio had developed so far didn’t do justice to blonde hair. At all. So research needed to be done and a hair model was found and her hair photographed in probably any appearance (even wet) and every possible lighting situation. Also commercials of hair care products served as a valuable reference “since they propagate the ideal on their packaging we wanted to recreate in the film.” Even a PhD student did extensive research on the subject and in the end a feasible shader was programmed:

First of all, hair comes in individual strands and strands that make up a volume of hair. Moreover, these strands have a top, a bottom and a body with an uneven surface, none of which is like the cylinder-representation of hair we all used to work with. So hair has a specular reflection, so far so good. And for black hair that’s usually all you see. Light hair also has a sub-specular portion that is a broader highlight in the color of the hair, whereas the specular highlight has the color of the light source. Then hair has a transmission value when lit from behind, and multiple scattering is important for light colored hair. Still that’s not enough because a hair rarely comes singly: There’s also a volume diffuse portion, that is backward scattering of the light and volume transmission, which scatters away from the light source through the volume.

Now if you add up those five components it looks good but not perfect, a certain richness is missing. This “richness” comes from the light bounced back onto the hair and ambient occlusion. Then, and only then, you are rewarded with beautiful blonde hair. But for eyebrows, fur and eyelashes you still need a different shader because the blonde shader needs a volume to work with after all.

Appeal

And another question: What constitutes appeal? What makes a person look appealing in a movie? How to light a character to look appealing? Mohit dug deep once again and looked through lots and lots of glamor-shots of actresses and actors of Hollywood’s golden age in search for unifying principles. And his answer is simple: “Cheat!”

The starlets on photos (and in movies) were always lit by a soft light, no matter what the rest of their environment looked like. Then there needed to be hue in the shadows, the dreaded “graying” of multiplying occlusion with diffuse passes makes characters look sickly. Since the eyes are the window to the soul, they needed special attention and always a specular reflection, no matter what. And as painters suggested, the cavities of mouth and nostrils should not go completely black but into a warm darkness. And unappealing colors (usually green bounces of leaves and grass) also make a character look weak and sick, so they had to cheat there as well. Finally a subtle bloom on the highlights never hurt anyone.

Look

On Tangled there were 19 look development artists that in the end produced over a thousand paintings of looks. And lighting was important to elevate the mood that was already there by narration, staging and framing. As Mohit showed some examples of lighting I could not help but remember the words of my mentor, Fraser Maclean as Mohit uttered them as well: “Put light where you need it, not where it realistically would be.” Indeed, the progression of shots he showed were lacking any continuity of the origin of the lights. “And here… well I don’t know where the light is supposed to be coming from, there is no window there” …but still it works perfectly in each shot. It went so far, that even the murals in the tower could be toned down or changed in opacity on a shot-by-shot basis. “Light shapes add to the drama.”

And so does saturation. In dramatic sequences the saturation got sucked out of the pictures, only to return almost fully later on.

Art-Directed Trees

Once you can art-direct everything, you have to art-direct everything. The R&D department provided a simple tool for Maya that allowed the modeling artist to draw a couple of curves and the plug-in would make a tree with branches out of it. Moreover, they also had a tool that would grow leaves into a pre-defined canopy-shape which was really fun to watch.
And since there are so many trees in a forest, the models were also switched to “brickmaps”, a RenderMan-term for low-res voxel-representations of a model, if far enough away and automated “stochastic pruning”, which meant that depending on the distance to the camera, not visible leaves would be automatically switched with low-poly models or removed entirely.

“In the end we generated too many trees and had to hide them in haze or atmosphere in post” Mohit admitted. That really was new to me, having accidentally too many in the final picture of something so complex such as trees!

Tangled was rendered with RenderMan (finally another Reyes renderer today, I was almost worried!), had 1380 shots and 55 lighting artists which, in sum, resulted in 9.01 million hours of a single lighting thread. “That’s more than 1028 years”. Ah, statistics. It’s like I tell you now that this blog post is already some 6000-odd words long.

Physically Based Shading

Ben Snow,
originally uploaded by Phil Strahl.

I used the short break to get back close to the front in row 2 and to empty my can of Starbucks “Doubleshot Espresso” to be all up an ready for Ben Snow’s and Christophe Héry’s Physically Based Shading at ILM¹.

Ben showed some definitions at first outlining GI, IBL and HDRI. “We’re not trying to reproduce reality,” Ben made clear, “but filmed reality” and described a little ILM’s history in their attempts to achieve this goal over the years.

The practice of trying to capture as much information from the set grew over the years in scope and professionalism as well. In the 1990′s they started filming and photographing light probes on set, 18% gray balls, to recreate the light later in the computer. But soon they realized that was not enough. Along came six photos in each direction for cube-maps and then the familiar chrome ball, a technique I never quite got to work for my own projects. Indeed, the chrome sphere’s reflections were rather low-res and you needed to paint out the photographer every time as well.
So in the “early days” ILM made heavy use of texture maps with painted in highlights and shadows, for shading they employed the Cook-Torrance specular model. Their light rigs were also pretty basic but could handle real world situations already rather well. Occasionally shadows grew really dark and some ways to cheat were e.g. spot lights or churning down the overall shadow opacity².

Then came Perl Harbor when they really needed to crank their depicted reality up a notch since ambient occlusion wasn’t enough. So they developed what they called Ambient Environment Lighting, which essentially is creating a pass for the ambient lighting via ambient occlusion: Rays are cast in a hemisphere around the surface normals, then a number of rays hitting other surfaces dictates the occlusion; and a pre-pass is done to calculate the average direction of the light. At least that’s what I copied from Ben’s slides. The film was a milestone nevertheless because Michael Bay, the director, was not able to tell the difference between CG and live-action-footage anymore.

What happened after Perl Harbor was boosting the quality consistently further. 8-bit images had served their purpose well (including in Perl Harbor!), but the need and time asked for floating point precision. Also, no mirrored balls would be photographed anymore in favor of photographed panoramas.

So a couple of years afterward came Iron Man and asked for the realistic depiction of, well, iron and metal that needed to match the practical suits on set. And a thing that had been troubling the folks at ILM was anisotropic highlights that appear on brushed metal.

Further, a new approach for taking HDRI panoramas was employed and meant taking a series of photographs from a tripod in all directions, although (as it appeared to me) not in a truly high dynamic range but only covering two two exposure brackets — too little from my experience but still it worked for them.

New Frontiers

And just as they thought they had mastered metal, along came Terminator: Salvation where it was impossible to cheat any longer and they had to move to a new paradigm for lighting and shading in RenderMan. ILM’s goal was to get a simpler, more intuitive and physically based system of lighting and rendering:

The quality of shading and lighting requred a BRDF-model that not only looked right but also acted physically correct in terms of energy conservation. In short this means, that the rougher a material is, the weaker is highlight gets, otherwise the energy (= light) reflected would be bigger than the light received — impossible³.

That also meant a normalized specular highlights with a more physically plausible specular falloff, so the further away a light source is, the weaker the specular highlight gets, depending on the roughness or the “Normalized Importance Falloff”: The intensity of the highlight falls off on rougher surfaces. For tight specular highlights like chrome or mirrors the light source has to get a long way away before dimming. The broader the speculars are, the more quickly they will dim. For this to work the light needs to have a physical size in the system, so no more point-lights or directional lights at ILM

“This was hard for everyone to adjust to and Christophe and I remember some very passionately fought holy wars” Ben remembered.

On the set of Terminator: Salvation the chrome spheres were back, but differently. Now they were moved and shot in motion so their reflection could be used on moving models. “We still don’t have a great way of capturing HDR moving images,” Ben continued in front of a turntable of a T-800 in front of a steelworks plate, “instead we shoot HDRIs with stable lighting and we also shoot our chrome spheres so we still get our FX, strobes, sparks, etc. And on top of that we applied some pyro elements shot on film and used them as reflections or area-lights in the scene.”

Iron Man 2

“We now had our tools that worked, they weren’t really mature but technically they were robust” Ben noted. For image based lighting ILM then used a graphical tool, the Environments Browser, to quickly define light sources within the image so they could be recreated with area lights. The match-move of a shot creates the environment and the HDRI panoramas get projected onto the geometry and you basically end up with a HDRI-mapped recreation of the set environment in 3d. That enabled the artists to render dynamic HDRIs from any position in the set to be used in image based lighting of the digital characters.

IBL via HDRI

But Image Based Lighting also poses some risks or at least things to watch out for. An important thing to be aware of is that IBL-lights are treated as infinitely away point-lights from the scene. A fact, that comes distractingly into play when CG-lights with a discrete position within the scene are meant to work alongside an IBL dome. Then it really takes an experienced photographer and visual effects team to produce good and usable HDRIs from the set which is essential for this approach. Last but not least the pipeline should support the the floating-point nature of HDRIs or else one might run the risk of losing dynamic range along the process when editing HDRIs.

On set ILM still photographs chrome and grey spheres but only as references. This only works properly if the film crew is in the habit as well and does not treat those shots lightly or worse, forgets about them. Since the VFX team later needs to match the lighting of the shot everybody liked on set, those references should be shot immediately after the director yelled “cut!”. “You wanna make sure that your spheres are as big as possible in the frame and you wanna make sure they are in the right spot. And you don’t wanna shadow or be reflected in the sphere” Ben reminded. Sometimes when the spheres are moved like an object though the scene, it is advisable to take some static sphere footage as well.

“On the set as soon as they say We’ve got it! then off you go and get the references.” These references, of course, should only be shot for shots that will have CG portions but if there is ever any doubt that a certain shot definitely will not require CG, one should still capture the references to be on the safe side.

“And be serious about it. If you don’t take this seriously nobody else on the crew will” Ben shared his experience with the audience and switched to his last presentation slide, titled “How we capture HDRIs” which I shall reproduce here:

• Canon 1Ds Mk3 with Sigma 8mm fisheye lens
• Nodal Ninja & Tripod
• Remote shutter trigger
• 0.6 ND (2-stop) filter (depending on how bright it is)
• 7 exposures, 3 stops apart
• Direct sun f/16, ISO 100, center exposure 1/32 sec

After a quick slide of acknowledgements Ben handed over to Christophe who would once again talk in a little more detail about the math-laden background behind the presented concepts.

Since my senior-high maths-teacher sucked out all the fun I ever had with mathematics, I wasn’t in the mood of trying to follow Christophe’s every word but I got the general idea:

Calculating reflections and IBL with the Monte Carlo approach requires a high amount of samples for a clean picture. Since the distribution of the rays is random, you end up calculating a lot of stuff you will not really see in your final rendering. So he came up with MIS, Multi Importance Sampling, that takes into account what the camera sees, that the light illuminates and identifies an area in the lighting-dome where rays have a probability to affect the final rendering, such bright portions in the HDRI used for IBL. With the same number of samples used for rendering, which are only weighted differently, in the end compose a much more pleasing result because you only calculate what you need.

To implement this into the renderer, the BRDFs and lights need to provide eval() and sample() methods, where eval() returns a color and a pdf for a given input direction and sample() that returns an array of directions and pdfs: For instance, in a dome IBL situation, these will be the vectors to the bright spots in the image.
I just copied this from a slide of Christophe’s presentation, so don’t ask me what it all means. The only thing I am pretty sure is that pdf in this context stands for probability distribution function and not for Adobe’s favorite way of storing their user guides.

ILM’s BRDF specification has to follow a number of principles too: The shaders now must be normalized e.g. energy conserving, they must “substitute” to ILM’s trusted but old Cook-Torrance model, they should be anisotropy aware and, as always, efficiently computed.

Their solution now is D-BRDF based on an yet unpublished paper by Michael Ashikhmin and Simon Premoze; the Beckmann distribution⁴ and no masking: the reciprocity term is simply 4.0 * V.H * max(L.N, V.N).

For the nerds among you I even noted the links Christophe’s presentation closed with:

• Eric Veach’s paper⁵
• Last year’s SIGGRAPH course
• D-BRDF

You’re welcome. Now where is my mind?

After this heavy information laden day I skipped nVidia’s panel discussion and went straight home to type up this blog post. who would have thought that it would take me effectively over two weeks to finish it?

Hey everybody! Sorry for not updating in quite a while yet again, but as you already know: When there’s not much on the blog going on, then there’s much work I’ve been doing. Amidst all the NDAs and top-secret stuff there’s something now from a current project I can show you. Read on if you’re interested in the current progress of the awesome project I am currently involved in and wanna see some eye-candy.

Freightliner Truck Spin from Phil Strahl on Vimeo.

Be sure to watch it in HD!

I’m not going to tell what movie this truck is from, but some of you might already know. I am rather proud of the shading and texturing which was done with simple Blinn-shaders in Maya (apart from the windows, which are a mia-Material). All modeling was done by Jonny Hennebichler by the way, who will be also responsible for the animation to come. And, boy, that’s gonna be awesome!

For the dirt specs and scratches I have three custom Photoshop brushes and I used mostly photos just as references instead of textures, so the only photographs that ended up in maps are on the axes (you can’t see them there though) and the tire rubber.

Since the truck is intended to be rather prominently featured in a 720p production it is rather heavy on details and map-resolution, in total there are 50 maps, mostly for color and specularity, some of them are in 4k resolution.

One frame of this turntable rendered in about 4 minutes on my quadcore, with medium quality Final Gathering enabled.

Since the lighting was done with Physical Sun And Sky I had to do a quick color grading in Nuke where I also added some grain so it’ll give you an impression of the look I want to achieve. It is not perfect, e.g. I chickened out with the reflections, they are too subtle, but in sum I am rather happy with it — I hope you like it was well.

Work in progress.

Another time it’s been way too long between updates. Again much has happened, I was too busy to author a thorough blog post of general interest. And honestly, I still am. So I will give you a short run down on what I did recently. And if you want to see some shading/texturing going on, to witness the power of the almighty bump-map that still does its job well and without the annoying purple eye-cancer-causing colors of a normal map, read the rest…

After posting the last blog post I…

• …worked on some station ID motion graphics as a compositor at Fresh FX. Lovely place!
• …took a week off, hoping to get anything done with my diploma thesis.
• Instead I am working on a rather secret project on lighting, shading, rendering and compositing. Oh and I got me a Canon EOS 5D Mark II. As soon as I produce something worthy I’ll let you know. But I already know how much I love that black brick already.
• …did an elaborate video tutorial for AEtuts+, that’ll go live this week or so.
• ..enjoyed two weeks of heavy duty Nuke compositing for Fresh again, complete with long hours and weekend shift.

Now I’m back to not writing on my thesis and continuing on the secret project, codename Trailblazer.

Some of my colleagues already know about Trailblazer but for those of you who don’t I’ll have a little riddle. Look below, this is a small part of the project I shaded today — w00ts for Photoshop! All I can tell you it’s from a movie that was released nearly 20 years ago. If you can guess it from the detail: Damnit, you’re better than good, you’re all knowing! So you’re either god, the anti-christ over the über-nerd. Either way I am happy that you spend your precious time reading this little blog entry!

License Plate Shading from Phil Strahl on Vimeo.

And don’t bother googling, it won’t help you here — hehe!

Because I had lost my Ratatouille poster last night I decided that the best thing to do was to attend another RenderMan presentation, get another walking teapot, get another RenderMan sample DVD and to fill out another survey sheet. In the comments section I apologized for showing up again only for a new poster. Sometimes you just got to be honest.

The Plan

The people from the jumping lamp studio

Morning in Stuttgart,
originally uploaded by Phil Strahl.

But instead of hearing the same stuff again about the integration of Massive into the Pixar pipeline by Ziah Fogel there was Afonso Salcedo, one of the lighters at WALL·E who did about 3 or 4 percent of the shots in the whole movie, “a good average”. I learned much about the way of lighting at Pixar, and how they dodge using ray tracing, the Achilles heel of RenderMan. For WALL·E the director wanted a 70s sci-fi movie look so Pixar got Dennis Muren over to set up some of the basic lighting for WALL·E in the atrium with a 70mm-film camera with anamorphic lenses and some WALL·E and Eve mockups in real size. They did tests on the DOF, the cushioning and the way lens flares and iris circles look. That’s what I love about Pixar: Total involvement and research for every project.

I asked Dylan Sisson (?) afterwards and he answered me quite sharp that they don’t because they want to keep the full artistic control over the light ¹. They even light characters and background separately. So they set everything by hand, about two hundred lights each shot. No wonder it takes them so long!

When the presentation was over I got myself a nice espresso at the lounge and returned half an hour later to the recruiting presentation which was in fact a Q&A session with all the folks from Pixar that were present in Stuttgart: Carlos Baena, Sharon Calahan, Dylan Sisson, Afonso Salcedo, David Munier, and Ziah Fogel. I seized the day and asked them a lot about anything that came to my mind, most importantly how they do their compositings and what software they use and whether they are looking for a compositor.

That hit me deeply. They don’t even have the job I was hoping to apply for.

After the session there was a buffet in the little room outside the seminar room and the audience mingled with the Pixarians. Paul from the 4th semester was interested to learn from Carlos how the animation of WALL·E’s crawlers played a role in conveying his character. I was still eager to get some of my questions answered by Sharon Calhan, Carlos Baena and Afonso Salcedo at one table – unbelievable. And silly me forgot to take a picture before I left for some presentations with “educational” value for me.

3D and Live Action in 3D. In stereo

Meeting Colleagues,
originally uploaded by Phil Strahl.

Stereoscopy is the buzz of the year, as I mentioned earlier. Disney, Pixar (yes, there’s a difference!), DreamWorks, ILM and Sony ImageWorks are working on features in stereoscopic 3D now and all of us artists in the industry either adapt or become extinct. I chose to adapt but was very skeptical about it.

Journey 3D by Bret St. Clair (from Meteor Studios) and Chris Harvey (from Frantic Films) was the first presentation in stereo ³. The two studios in Canada have been working on The Journey to the Center of the Earth 3D starring Brendan Fraser and had to handle a live action/CGI feature to seamlessly blend together – all in stereo.

The two of them talked about their struggles and problems and seemed as if they were telling a terrible tale of how they had survived 9/11 and Katrina. They had to tell so much in such a short time that I noted down like crazy what I felt was important. And there was a lot, starting from the camera rig on set. They were using two different rigs, one that allowed to adjust the interocular separation ⁴ even to zero if they had wanted it to thanks to a 45° semitransparent mirror (which produced differences in distortion and lights), the other rig had an interocular separation of 2.7″ as opposed to the 2.5″ space between the average viewers’ eyes.

Zooming, for example, was nearly not possible because the two cameras had to be in totally in sync which was not possible in live action shooting because even an offset of the tenth of a pixel makes a difference to the perceived depth.

Matte paintings should be executed at least in 2.5D, but it is much better to project them onto geometry which shouldn’t be too low-poly because you just can’t fake it in stereo. And don’t paint haze, fog or atmosphere in your matte paintings because that all needs to be done in 3D.

You can’t make a 3D film into a stereo 3D film at one point, you have to be certain about the presentation when you’re working on the concept because from that point on, nearly everything should be executed in stereo, especially the layouting and animatic. Artists that have only worked on non-stereo pictures before (which is the majority of us at the time) tend to cheat to get the image an aesthetic quality instead of being 100% correct in a three dimensional space.

Create optimal viewing conditions. From the emphasis put onto that statement it is the most important one. You have the same projectors on set and in your studio as you will have in the theater, you have to have at least a 20 ft screen for the projection and there should be one 2k stereo-projector instead of having two separate projectors because you will have to stop the footage, scrub through it, etc. to review your work.

Stereo means to double everything: Storage space, amount of footage, time for trouble shooting, compositions and rendering time, when you’re not working with some tricks: In the case of Journey they were able to have the second image rendered in only 20% of the time of the first one by caching a lot of render-data so only the occluded geometry had to be re-rendered. Pretty clever and it saved them a lot of precious rendering time. They used Gelato by nVidia for that, but I have no idea what it exactly does.

And now for the dreaded, feared and gruesome focus on matchmoving in stereo *insert 50′s horror film music here*. Even for ground planes it had to be done absolutely correct to every 0.1 pixel otherwise something would’ve been off in the final depth perception. A tracking cube helps you much in those cases because the perspective its clearly visible at all times. After tracking you should test it in the theater-situation before continuing because it has to be absolutely correct. The guys developed a way of tracking only one view of the stereo and then calculating the correct track for the other “eye” because tracking them separately didn’t result in absolutely the same movement for each eye.

Keeping as much meta data from the camera as possible in each shot. You might not use it at all times but sometimes it really saves your shot.

Another thing to know is that distortions work differently. Instead of having a centered distortion for every eye, the distortion should be in the middle between the eyes, making them off center for the individual camera footage. Lens artifacts should be matching in too: It is distracting, if you have a lens flare in only one of the views or that highlights don’t match and hence “flicker” when looked at. If you can’t avoid this on set you have to either remove the highlight in question or digitally double it for the other eye.

When it comes to compositing you want to have both eyes’ footage in a single composition, although it doubles the complexity. But you can make corrections to one eye where needed and have them share nodes and effects at the same time. Ideally the compositor should be sitting in front of the 20 ft so s/he can always check what it’ll look like in the theater. “Awake” for Fusion does the job ob having both eyes in one composition, by the way.

When evaluating stereo depth you should always revise shots in sequence so you can avoid too harsh differences in depth that the audience’s eyes don’t need too long to adapt each shot and might miss the important action.

Oh, and bump maps should be traded in for displacement-maps!

3D in 3D

Stereoscopic Depth as a Storytelling Tool by Mark Empey and Robert Neuman from Disney Animation was a very insightful presentation because there literally were worlds between their stereo-footage and the previous presentation. Disney showed off a five minute long montage of the stereo version of Meet the Robinsons and the depth perception worked just fabulous, seemed harmonic and believable. The first seconds were terrible, the following 20 seconds were okay and after the first minute I never wanted to watch plain mono footage ever again. Journey looked 3D-ish but not quite that pleasing. Even I could tell, that Disney had mastered 3D as they had mastered 2D. And they gave an insight about their way:

First of all you should avoid using stereo like it was used in the fifties and eighties:

Making it comfortable for the audience is the most important thing. If they get headache, hurting eyes and nausea from watching your stereo movie they will never want to watch one again. So you should keep the interocular distance of 2.5″ in mind. That’s not so much important for near objects that are “floating above” the screen, but for those who are beyond infinity: When looking at objects that are on the horizon both eyes are exactly parallel. But with the use of floating windows (more about them below) it might happens to have the audience look beyond that point where the eyes aren’t looking just parallel, they’re looking outwards which is painful because mother nature never wanted us to look that way. So avoid having offsets bigger than 15 pixels towards infinity.

Just like using a color script it helps much using a depth script where a value from one to ten (sometimes eleven) tells the stereoscopic supervisors how deep a sequence, scene, shot has to be. Neuman (or was it Empey?) said that for him stereoscopic depth equals with emotional attachment to the character. So a depth script is essentially a script where the arch of emotional depth is traced. The closer you want your character to connect with the audience, the more you push her/him out of the screen (not out of the floating window, to be described soon!). For emotional distance keep your character “behind” the screen.

Sometimes you want to have objects come out of the screen very close. It’s okay if you do it in the middle of the screen but when one of the stereo images is being clipped by the screen frame, then a “window violation” occurs. When this happens a viewer perceives the presented images much flatter because her/his brain needs to constantly decide between two conflicting different depth-cues. So what do you do? You produce a “floating window”.

The floating window is created by unequal masking of the different stereo-images, e.g. you give the left image a thick black border on the left and the right image a thick black border on the right. This little trick lets the screen in the theater appear as if it was floating in front of the action, depending on the thickness of the black border because the viewer always associates the floating screen with the physical one.
You can even have the floating window rotate and tilt, all with diagonal masking. Disney uses a Maya plug-in that displays for the stereo camera rig in the viewport the near plane (where nothing can be any closer without risking eye strain), the floating window transformations, the screen plane (where the physical screen in the theater is located) and the far plane (which defines infinity) and can be controlled by eight values.

DOF should be limited and only applied to objects in the background (“behind” the screen). For objects in the foreground you should deemphasize their importance with lighting.

Don’t toe in the two cameras in your rig, because this results in vertical mismatching and is uncomfortable to view. Instead keep the cameras parallel at all times and use only a different portion of their perspective correction lens works.

At last it is important to have your characters round enough to avoid that your characters look like cardboards which occurs when the roundness of a character drops below 20%. This occurs when using telephoto-lenses where a lot of depth-information is eaten up by blank space. One way would be using a wide angle, but a better way it using multiple camera rigs, one for the foreground with all the necessary depth information and one for the background. So you can skip the blank space between character and background and use this range to enhance the roundness instead.

Disney used all these techniques for their upcoming feature BOLT and from the very raw work-in-progress material that was screened I can say that it works marvelous.

French Stench

Sorry about that headline, I just like cheap rhymes (but no hip-hop!). The presentation titled Effects Tailored to a Director’s Vision by Pierre Buffin and Francois Aubaque from BUF Compagnie sounded rather interesting, because BUF use only proprietary software since they started in the eighties and have grown in technology as well as in style and expertise. They did the effects on Fight Club, Panic Room, Silent Hill, The Prestige and any Astérix and a heap of others. So I was quite thrilled to learn about their work with the directors such as David Fincher.

As it turned out they were talking English with such a terrible French accent that it was hard to make out what they were talking about. But instead of telling much they just rolled making-of after making-of the different productions and just said a few words between them. I was quite disappointed because I can watch all that making-of-stuff alone on any special-edition DVD of the given movie (even without the accompanying French songs). So I left after twenty minutes for another caffeine flash to get ready for Carlos Baena‘s much anticipated Realistic vs. Stylized Human Animation presentation.

He showed his work on Star Wars Episode II as example for realistic animation and some shots of Skinner in Ratatouille. The shots he worked on are also on his website. He also showed a short animated clip from the Iron Man Trailer, the shot with the iron man’s reaction as the tank misses him: He dodges, leans back more than he had anticipated so he has to take a step back to keep the balance. He lifts his arms, looks back where the shot came from and shoots back from the gun on his arm. The rebound travels up his arm and reaches his head an chest after a couple of frames, so he shakes a bit, the left arm extended into the other direction to keep balance. Now was that a thorough description of a little animation or what?

Then he showed some reference footage he uses as an animator, a DVD with slowmotion footage from the Olympics in Japan some time ago where you can see most of the actions from the side or the top. Some other references are x-ray films, e.g. of a man raising his arm.

He looks into the audience and continues honestly.

He showed some more footage in slow-mo, of a fist hitting on a table, where you could see the muscles and the skin ripple on the impact. That’s the stuff you should be watching all the time when becoming a character animator!

In terms of stylized character animation Carlos dug up some early tests he did on Skinner, the cook from Ratatouille who was based largely on Louis de Funés, my favorite French comedian. Just watch his performance in Jo and you’ll see why the folks at Pixar decided that Skinner became too French.

Unfortunately the amount of coffee I had pumped into my veins that day made me rather immune to any more caffeine so I had troubles staying awake during the whole presentation. When Carlos was talking something about Woody’s Arms in Toy Story I drifted away for a couple of seconds, or a couple of minutes — I don’t know.

The presentation titled Bringing Nuke and Maya Together by Andreas Frickinger and Daniel Hasenbring from Miromar was held in a way too small room for way too many interested people so I was among those who sat on the floor, when the two Germans showed their tricks of rendering a Mental Ray mia_matrial in a crapload of passes and having a script rebuild the shader in Nuke again, where you can tweak on every nut if you ever wanted to.

The most stunning part, however, was the possibility to re-light your rendered scene within Nuke because their pipeline also exports UV and World coordinates. The only drawback is that your Nuke-lights will not cast any shadows but for subtle re-lighting it’s perfect.

Even some of the scene geometry can be viewed in Nuke’s 3D view — it’s crazy how such an amazing workflow-demonstration came off like a high-school presentation of two totally not entertaining people that seemed not too well prepared either.

Afterwards I talked with Kerschy a little who also witnessed the presentation. He was so amazed about the whole topic that e decided to write his diploma thesis about how to get from Maya to Nuke like the guys from Miromar.

What remains as item on a list of impressions:

Morning Impressions,
originally uploaded by Phil Strahl.

• My first real meal this week, enjoyed at one of the local McSteakhouses, Maredo.
• Strolling around the heart of Stuttgart without knowing a place to go.
• My visit to the Schräglage-lounge with lots of goofiness, caffeine and laughter.
• Another day without warm water at the hotel.

What I have learned today

• That you can’t cheat when working in stereo.
• That zooming is not possible in live-action stereo
• That subpixel-accuracy matters to the tenth of a pixel in stereo
• That matte paintings should be at least in 2.5 D when working in stereo
• That you can’t paint haze into the matte paintings when working in stereo (= faking it)
• That even the layouting of a stereoscopic shot/scene/film should be done in stereo
• That you should be able to view your dailies the same way your audience will watch the final stereo picture.
• That you should create the same conditions on location for viewing stereo dailies as your final audience will have.
• That camera shakes can be used (moderately!) to hide stereoscopic issues.
• That stereoscopic 3D-rain looks fake in Journey. Really fake, just big blobs dropping.
• That when working in stereo you need to double nearly everything in your production calculation.
• That matchmoving is nearly impossible for live action stereo footage.
• That using a cube for tracking stereo footage is much better than using single tracking marks without any depth information.
• That you should collect as much meta data from the cameras as possible when working in stereo.
• That distortions in 3D share the same center, which lies between the “eyes”.
• That lens artifacts should apply to both eyes. If they don’t match make them match.
• That a composition for stereo shot should be done for both eyes in one composition.
• Stereoscopic depth should be kept persistent throughout a sequence.
• That you should use displacement maps instead of bump-maps for stereo pictures.
• That you just can not cheat when working in stereo!
• That you shouldn’t use stereo for gimmicks
• That you have to make watching stereo as comfortable as possible for the audience.
• That you should avoid “beyond infinity” focusing on your audience in stereo. 15 pixels at 2k are enough!
• That you should use a depth-script when working in stereo.
• That perceived depth can equal emotional depth.
• What “floating windows” are in stereo and how you can work with them.
• In stereo DOF should only be used for the background, for the foreground deemphasize with lighting.
• That you should use a different portion of the image circles of parallel cameras in stereo instead of toeing them in.
• That you can counter cardboarding in stereo by using two camera rigs, one for the foreground, one for the background.

The idea was quite simple: I wanted a gray or asphalt colored shirt with a white DMC logo on it because working in Maya on a DeLorean time machine for so long leaves its marks, especially on somebody who as is easy to manipulate as me.

If you try to find that shirt via Google you will spend a while until you end up at the DMC website where you can buy it for $18.95 plus $49.42 (ahhh!) international shipping fee. Seventy frickin’ bucks for a quite simple shirt is waaay too much! “I can make one myself for less!” I said to myself. And that’s how I started.

Yesterday I had a brief chat with my colleague Dansch Übleis who is famous not only for his Maya skills (especially workarounds and cheap time-saving fakes) but also for his hands-on creativity such as, but not limited to, printing t-shirts. Because I wasn’t eager for silk-screening and the hassle with it he suggested printing the logo onto a cheap ink-jet transparency, cutting out the shape with a paper knife, sticking it with tape on the shirt and filling it with Marabu brand window colors. “It’s as easy as that!” he said and I believed him.

Shortly after I was on my way to a shop of the Libro stationery chain to get my assets. Along with 20 floppy disks (don’t blame me — I just enjoy writing on my ancient ThinkPad from 1996), two DVDs, 1000 sheets of paper, a white ball pen and Final Fantasy XII I bought what I needed for my custom DMC logo shirt: A small can of Marabu fun&fancy window color (white) for 3.69 € and a pack of 20 sheets 3M ink-jet transparencies for 17.99 €.

At home I fired up Illustrator and quickly drew the shape of the DMC logo, accompanied by a line of type reading “De Lorean Motor Cars Ltd” and printed everything in gray onto one of the transparencies. Cutting out the shapes (especially the relatively small line of type) took me half an hour before I was ready. Before taping the stencil onto the shirt, an asphalt colored American Apparel 50/50 model, I applied a lot of paper-glue on the backside of the stencil so the thin bridges would stick to the fabric later that no color would spill where it shouldn’t. I applied plenty of the window color with a soft-foam form until the shirt was majorly soaked — I put it on the radiator so it could dry.

A couple of hours later the color had dried in and was anything but opaque nor white. I was a little disappointed about the outcome and pondered for a while how I could save the shirt. Luckily I still had a fresh can of white textile color so I decided to give it a try and stamped the new color over the pale window color with the foam-form again. This time it looked way better. Again I let it dry.

Another couple of hours later the textile color was opaque and dry and I was eager to see the result. I removed the tape from the stencil but it stuck to the shirt like crazy thanks to the thick color. I literally had to tear my transparency off the shirt which left my stencil’s filigree parts somewhat damaged; hence I won’t be able to reuse it if I ever want to. Bummer.

But apart from the slight off-centered orientation (I still have troubles aligning things with the naked eye since kindergarten) it looked really neat and much better than I had anticipated: No color bleeding, only silk-screen-like crisp edges and a thick and well-textured color. I don’t know if that’s the effect of the window-color base or not, the only thing I know is that I was happy making something myself and saving money with it too. In total the print (without the shirt) cost me as little as about two Euros — I spend more money on coffee everyday! I hope it will stay looking that good after the first washing because I cant iron it in.

I guess I will be trying out some more unique t-shirt prints soon. But at first I have to finish my work: Shading the DeLorean time machine in Maya photo-realistically. Maybe the next print will be a DeLorean too…

Software used:

• 3D Studio MAX 6 (“3dsmax”)
• Brazil r/s V1.2.21 Plugin (“Brazil”)

Recommended Skills

• This tutorial is very basic and mainly for beginners starting with 3dsmax. So don’t worry.
• You should only know how to create new objects in 3dsmax and have Brazil properly installed, which shouldve been the toughest part however

The Goal

• Creating a realistic looking gold texture within 3dsmax.
• Gaining basic knowledge about the Material Editor and Brazil’s rendering settings
• Optimizing the rendering process

Let’s do it!

1. Start up 3dsmax and set the rendering engine to Brazil if it isn’t your default engine. If you already know how to do this you can continue with 7
2. Open the render settings clicking this icon on the upper left or hit F10.
3. Now scroll down to where one bar reads “Current Renderers” (“Renderer zuweisen”) and click it, so it expands. Scroll down until you can see the three assigned renderers for “Production”, “Material Editor” and “ActiveShade”. If all read “Default Scanline Renderer” you have to set the Production rendering to Brazil.
4. Assuming you’ve properly installed Brazil click on the dots next to “Production:” and highlight “Brazil r/s V1.2.21” in the pop-up selection box and hit OK.
5. Notice that now also the Material Editor’s renderer is set to Brazil, which is bad becase Brazil doesn’t support it. So we have to change it back to the “Default Scanline Renderer”.
6. Hit the yellow button depicting a closed lock just right of the “Material Editor”. The icon should now show turn grey and view an open lock – the renderer should change to “Default Scanline Renderer”. Check the picture below to see if you’ve done it the right way. Don’t mind the German descriptions.



7. Close the Render Settings by clicking on it’s X-button in the very upper right.
8. Now open the material editor via hitting M on your keyboard. If every slot reads “not supported” its rendering is set to Brazil. In that case continue at 1).



9. Click the “Standard” (1) button and select “Brazil Chrome” (2) and hit “OK” (3). Now we’ve got a pretty neat chrome material but not at all golden. Note that a reflecting metal doesn’t have a color itself, it rather filters the reflected environment in a characteristic way – so forget all of the “Gold Diffuse Textures” you found on the web. Gold only reflects the yellow and orange light of the spectrum.
10. Rename the material from “01 – Default” to “Gold” or similar.
11. Below, click the white rectangle right of “Filter” and a color-picker palette should pop-up. Select a warm yellow tone. I used the following settings: Red 255, Green 207, Blue 23. Click “Close”.



12. The “Glossy Reflection Control” section. First of all, what does it mean? Glossy reflection control determines how diffuse the reflection on the material is. It takes a lot longer but looks more realistic if turned on.



I rendered the above samples on a 2GHz Intel Pentium M with 1GB of RAM in highest quality with Global Illumination (GI) turned on. The little stopwatches indicate the rendering time in minutes.

For the gold-material I chose to keep the initial glossiness value of 80. If you want it blurrier, decrease it, if you want it crisper, increase it up to 100. In terms of rendering performance the glossiness value doesn’t really matter. Only if it is below 40 it slows everything down a little.

Also important for the rendering time is the sample rate. Initially it is set to 10 which guarantees a pretty smooth surface but takes quite long too. I chose a value of 4 this time, but it depends: If the golden object is very close to the camera, I would boost it up to 10 or even 13. If it was only in the background, a value of 2 or even 1 would be sufficient.

Below there are four enlarged examples of the cube’s reflection with different sampling rates of the glossy reflection control. Note that the value in the stopwatches indicates the time it took to render the whole image, not only the enlarged cutout. Glossiness Level was 50 in all four examples.



Please note: The graininess is also dependent on Brazil’s Image sampling rate and last but not least the Global Illumination Shade rate. For a detailed walkthrough see addendum A.
13. Still in the Material Editor scroll down to the group called “Highlight Parameters”. This is where we define what the highlight of our material looks like.
14. Click on the rectangle left of “Specular Col.” and pick the same warm yellow tone as above, maybe a little more yellow. I used the following settings: Red: 255, Blue: 236, Green: 23
15. Below there’s the value “80” next to “Level”. This will define how bright the highlight will be. I exaggerated it a bit and set it to 200, so the material will reflect a very bright light white instead of the orange we’ve defined for the specular color, just because it is so intense.
16. Now click the long gray button reading “none” right of “Glossiness”. The “Material/Map Browser” will pop up another time. Select “Speckles” (“Flecken”) and hit “OK”. Why so? I want the gold to look a little like car paint and with the speckles we simulate the encapsulated metal particles underneath a perfectly smooth surface.

The following settings kept me busy for days until they looked right, so you just have to trust me if you want it to look like my texture; or play around with the settings a little yourself.

17. Under the “Speckle Parameters” set the size from 0.1 to 2. Pretty big speckles you might think, but we aren’t finished yet.
18. Click the gray map button reading “None” next to “Color #2”.
19. Again the “Material/Map Browser” pops up where we select “Speckles” – again.
20. Here keep the size of 0.1 but change the color of “Color #2” by clicking on the white rectangle to open the color picker dialog box.
21. Here change the value of “Value” from 256 to 188, or enter 188 for Red, Green and Blue.

Now what did we do that for? We balanced the amount of the speckles so that it still looks natural. I played a long time with it so I know what I am talking about. To get a better understanding of what we just did save your work so far and try playing around with different gray tones for “Color #1” and “Color #2”.

>Important to understand is that in 3dsmax there are hierarchies in the material editor. In our case the base material is “Brazil Chrome”. On the first level below there is the fist “Speckles”-map for the glossiness of the “Brazil Chrome”-material. And even below, on sub-level two, there is the “Speckles”-map of “Color #2” of the “Speckles”-map for the glossiness of the “Brazil Chrome”-material. Sounds complicating, but once you are used to it it’s very practical because there is no limit in sub-levels.

If you want you can interlink thousands of maps into thousands of sub-layers.



To get up one hierarchy hit the “Go to parent” (“Gehe zu übergeordnetem Objekt”) button which can be found far right on the toolbar below the swatches (the spheres with the textures). The picture below describes the hierarchy a little.



22. Now hit the “Go to parent” button until it gets gray which means there is no more level above and see if your settings match those below:
23. 

24. That’s it! You only have to assign the material to an object in your scene. You do this by dragging and dropping it from the swatch in the Material Editor to the object.
25. Adjust the render settings (see addendum A!) and hit F9 to render.



Maybe your scene looks a little like mine on the picture to the left and now you wonder why everything is so dark and not really golden. Because, just as mentioned above, gold only looks like gold when there’s enough to reflect. And in a scene with just one spotlight there ain’t much. You can either build a huge scene around it or already have, or you can mock up a softbox-setup like in a photographer’s studio. I photographed a lot reflective objects and it is pretty hard to make it look like metal.

But especially when you need to render just a golden logo you don’t have the time to model a complete softbox setup. That’s why we create a nice reflection just for the gold material in five minutes or less.
26. In the “Top” viewport create a sphere or geosphere big enough that your scene including lights and cameras fits easily in. This will be our environment. Of course, you could do this in the “Environment” dialog, but then it wouldn’t be so easily controllable.



27. Now back to the Material Editor. Select a gray sphere from the swatches and rename it to “Environment”.
28. Click the rectangles next to “Diffuse” and “Specular” and make them both completely black by entering 0 in for “Value” in the color picker.
29. Also set the “Specular Level” and the “Glossiness” to 0, just to make sure.
30. But set the value for “Self Illumination” to 75. You can increase or decrease this value later, when you think the environmental reflection is too weak or too strong.
31. EDIT: Be sure to click the “2-Sided” tickbox unless your sphere will have no inside and hence will leave background and reflections black. Thanks for that, jintriag!
32. Now click the little grey square just right of “Diffuse” to open the “Map/Material Browser”. Now select “Gradient Ramp” (“Verlaufsart”) and hit “OK”.
33. Now you can see a smooth ramp from black to white. You can add a color by clicking somewhere into the gradient and a green key appears on the lower side. Double click the green key and the color picker pops up where you can set the color for the key’s position.

To delete a key click it and drag it left or right outside the gradient ramp. It will turn red and the mouse cursor will change to an upside down arrow with a trash bin. Release the mouse button and the key is deleted.

You can also click into the gradient and drag the mouse. This way allows you to pick up an value and drag it around inside the ramp.

It’s pretty easy so you won’t have to play around for long until you’ve figured out how it works.

34. When you feel familiar with it, try to make the gradient look like mine on the picture below:



35. Above, in the “Coordinates” section, make sure the “W” angle is set to 90 degrees, like in my picture. It’s pink for better conspicuity.



36. Great! Now apply your material to the big sphere in your scene, again by dragging the swatch onto the grid of the sphere in a viewport. But we only want to affect our reflection and nothing else.
37. Close the Material Editor by clicking on the X button in its very upper right.
38. Right click on the sphere in one of the viewports and select “Properties”. What pops up now may seem rather complicated but we only care about the tickboxes in the group “Rendering Control” (“Rendersteuerung”)
39. Untick every tickbox in the group “Rendering Control” except “Renderable” and “Visible to Reflection/Refraction”
40. Click OK – You’re done!
41. Click into your “Perspective” viewport and hit F9 to render your scene – now with proper reflections on your gold material.

Thank you very much!

Addendum A

We will only focus on the adjustments that matter for the output quality of the gold material.

1. Hit F10 to open the “Render Scene” dialog box
2. Click on the tab “Renderer”
3. Now you can see a long list of sections with tags such as “Brazil: General Options”, “Brazil: Render Pass Control”, “Brazil: Image Sampling” and so forth.
4. Click on the Tag “Brazil: Image Sampling” to expand that section. Most important for us are the “Min Samples” and the “Max Samples” values.

These values define how many samples per pixel are calculated during rendering. If it is below 0 then only a fraction will be rendered which results in a very rough but faster output. The min value set the minimum of samples per pixels; the max value defines the maximum of computed samples, e.g. for hard contrasts or edges.

There are three presets already set in Brazil, and accessible via the Buttons “P1”, “P2” and “P3”. The worst image quality offers “P1” with the fastest rendering. The line “Min [-8 x -8] Max [1 x 1]” tells that one rendering sample serves 8 pixels, utmost 1 pixel.

For a better looking preview hit “P2”. Beware that although it looks pretty nice the “Min Samples” value is below 1 and results, especially in animations, in flickering details.

“P3” offers with “Min [2 x 2] Max [4 x 4]” production quality which we want to achieve, but also takes very long to render.

Why more than one sample per pixel I hear you asking. The answer is simple: Quality. The more samples you got for a pixel the closer you can estimate its color. Edges will be very pixelated and either one color, or the other. This is also called “aliasing”. The more samples per pixels you got, the better the renderer can choose in order to make edges smoother (=”Anti-aliasing” or “oversampling”). For a detailed description of anti-aliasing visit this Wikipedia site.

Of course, you can change the min/max values after selecting a preset to your needs. Especially when you’re rendering high quality stills, you might even boost the “P3” settings up to Min [4 x 4] Max [8 x 8].

5. Click “P3” and close the “Brazil: Image Sampling” section via clicking on its label again.
6. Scroll to the section labeled “Brazil: Luma Server” and expand it, yes, by clicking on its label.

In this section everything concerning lighting is controlled, from direct to indirect lighting, caustics, even global illumination (GI) and sub-surface scattering. Don’t worry, we are only interested in a very view settings.<

7. Enable the indirect illumination by ticking the box “Enable” just under “Indirect Illumination”.

Indirect Illumination adds a great deal to a scene’s realism. It simulates the light not only emitted from direct scene lights but also the bounced light from other objects.

Indirect Illumination example: A spotlight on a red wall in a room would result in the whole room being lit by the bounced soft red light from the wall. Without GI you would only see the spotlight on the wall, the other side of the room would be completely unlit. More on Global Illumination on this Wikipedia site.

Of course, indirect illumination slows down the whole rendering, but you can define just like with the Image Sampling Rate how you like to sample the GI.

8. In the group “Global Illumination” you can set the min/max sampling rates for the indirect illumination only. For a fast preview the values “min -4” and “max 0” are okay, but you can expect a little splotchy results, which look like a very compressed JPEG. For a better preview or even production I’d say staying below 0 saves you time, I suggest “min -2” and “max 0”. Everything above 0 is pure luxury but looks stunning.

There’s also a tickbox for locking the sampling rate to the image sampling rate, which is quite practical for previews.

Before rendering your final, you should test a lot the performance with a very small resolution. Don’t waste a lot of time rendering details, nobody will see.

I hope this tutorial was useful to you. If you have anything to add or to comment on just do it – write me a reply below!