FMX flags,
originally uploaded by Phil Strahl.

Usually I would have jumped into the conversation of the fellow fmx-attendees but my internal computer said “No!” because it runs on caffeine and was dangerously low on it. So as soon as I got downtown I went to Dr. Starbuck for my medicine before attending the day’s first lecture.

Peter Plantec dropped a few words about today’s special program on virtual humans, after all he had written a book on the topic. It’s almost safe to say that finally we have made our way out of the uncanny valley. Almost. There still are occasional outliers but Benjamin Button proved, that it could be done.

And Digital Domain was spearheading that development as Matthias Wittmann’s presentation Combining Ages — TRON Legacy showed where he talked about the development and realization of Jeff Bridges’ digital age transformation on screen.

Head of Development

So how do you start such an endeavor? First you make a physical life-cast of your actor which gets you a full cast of the head (also the back of the head), good detail of the wrinkles, a sense of the underlying bone-structure and position of the ear holes, all that won’t change over the years — as compared to sagging skin. The downside of this method is that the person has to sit still for half an hour. “And when you sit still for so long you relax and your jaw drops. Like in many of you right now, your teeth-rows aren’t touching. And that’s a problem when you want a perfect model.”

Another time Jeff’s face was reproduced, that time it was a LightStage scan of his face. “LightStage has a really high resolution but you have to sit perfectly still for five seconds and look into bright lights. So inevitably your eyebrows narrow and even very subtle movements during the scan distort the geometry a bit.” But they already had the brows right from the life cast, so the artists could combine it with the jaw from the LightStage scan and a modeler crafted a clean digital model in Mudbox with proper topology.

Now they had the old-Jeff head. This was needed because the solver of the tracked footage recorded from today’s Jeff Bridges with four head-cams on set would only deliver correct results if applied to the old-Jeff model which would drive the animation on the young-Jeff head.

This head was modeled after the scanned geometry and countless reference photos, movies and videos depicting a young Jeff Bridges. This young-Jeff model went through countless back-and-forth stages of “Are the wrinkles there too strong? Take them out” to “Now he looks too soft, put them back in!”. In the end the animation-model of the head had 23,088 polygons in the Maya viewport. The rendered model was composed of 369,408 polygons but was lacking displacement mapping of very fine details such as the pores, which got hand-sculpted, in the model as well. So in the end, with displacement mapping the model had 5,917,916 polygons.¹

And the work on the model was everything but over: For the animation-blendshapes Digital Domain used a FACS-based approach for which they needed to capture Jeff Bridges as he flexed single facial muscles as good as possible independently in front of a scanner. This performance got recorded live with a Mova Contour scanner that offered a realtime scan of up to 60 fps, although 30 fps were sufficient. The realtime scan was not as coarse as a LightStage scan, naturally, still the high-contrast make-up on the actor’s face provided the scanner with roughly 700 tracking points that were translated to vertices to drive the motion on the geometry, although lips and eyes were missing. The data was good enough to select on a per-frame basis the best “take” of single muscles that could be incorporated into the blendshape models. A talented modeler also took care of the transitions to areas the scans didn’t cover, such as ears when moving brows. In the end the animation-rig consisted of more than 200 blendshapes two modelers worked on for half a year. And almost as sophisticated as the rig was the GUI for the animators. “The rig is really well done, so you can’t break the model, no matter what blendshapes you mix and put on top of each other”. Still, that process was never finished and constantly new shapes needed to be added over the course of the production.

With the expressions now possible the young-Jeff head was posed to match various expressions of the photo and video-references of Jeff Bridges to see whether they would hold up. Only if it worked there, it was feasible to work on shots of the movie.

Approaching a shot

On the soundstage, Jeff Bridges’ facial performance was recorded with four cameras on mounted on a helmet that recorded his face with tracking markers from four different angles. Occasionally a witness cam filmed his performance from a fifth perspective, “although 90% of the time there was no room or time for a witness cam. When I suggested that, you know, even filming Jeff with a small handy-cam or cell-phone camera would suffice, the crew just shrugged that they couldn’t do that because they weren’t in the [cinematographer's] union. So we had to work with the head-cam footage most of the time,” Matthias recalled.

Instead of moving the skin according to the tracking markers on Jeff’s face, Digital Domain used a different approach: “Our solver tries to use the existing blendshapes to match the points as close as possible, so the animators can tweak, add and adjust the generated blendshapes,” although movement on eyes and lips is not recorded too well, so every shot is a lot of manual labor of a talented artist. Further, “there is no solver in the world that can correctly figure out what’s skin-motion and what’s bone-motion.”

The eyes proved to be a critical factor also in this production. The blendshape-controls around the eyes had to be refined and worked on by the two modelers with special care. Even simple blinking needed a proper deformation in the face. The eye-lines always came from John Reardon, Jeff Bridges’ body double on set, since it was John interacting with other actors and the environment.

John, the body double, would perform what he observed from the take with Jeff to reenact to the best of his abilities. His movements would get motion tracked so that the digital head could be popped on. Once the animation was done on the head, a preview was rendered with sub-surface-scattering so it could be evaluated how it would hold up with light and shadows; then with the hair simulation and if it looked right there, it would get rendered for the final shot, composited and color-graded.

But the animation was not just fixing what the solver couldn’t do, there was much more to it: For example the many “Jeffisms”, as Matthias called it, subtle expressions or looks, that the body double missed to reenact. “The problem is, that you can’t change the head motion much, since body and head are a unit and breaking their motions apart would look just wrong. This really opens your eyes on how much verbal communication happens through body language.” Still, sometimes the animators needed to add a little nod on a specific word in the head-animation or twist it differently.

Then there was the problem of a swimming or floating head. “We sent it back to tracking and what we got back still looked floating. We looked at the footage again and whereas Jeff was speaking his lines rather serious, his body-double played it much more casual and relaxed and his head just wobbled.” So the animators let the digital face smile a little more so that the lightness of John would be carried over. So sometimes you had to adapt the head to the body movement. “I can’t remember how often shot like this one”, he pointed at the bobble-head-shot looping on the screen behind him, “were sent to tracking back and forth and came back almost the same because the track wasn’t the problem.” In the lower left corner the version-number indicated that we were shown version 47 of it.

Occasionally the production team even rendered the head from the positions of the four motion-capture cameras with the same distortion they produced, “just to prove a point, since everything looks different when lit, moreover, Jeff’s lips look really big and different than from the shot perspective. This way the decision-makers could compare the original and the animated emotion one-to-one.”

Headworkers

Not that often, but still, the team encoutered the opposite problem, that there was too much movement in the face, or some “Jeffisms” were happening that a young guy just won’t do so that needed to be taken out. Something that did happen a bit more often was edited reference footage, so that Clu would deliver his line angrily, only to laugh in the end, such as with the line “Where are you now? Ha-ha!”. In those situations the animators had to manually blend the different expressions together as plausible and natural as they could.

During the animation an HDR image of the set was used for a quick lighting reference, but it also happened that the lighting changed drastically once the head was positioned in the scene. In one instance the head was not lit from above but from below. “A smile lit from above doesn’t need to be that big to have the same impression than when the face get lit from below like it was in this case.” Oftentimes it was not easy to tell whether the animation of the face was dead-on, so an intermediate rendering of a Lambert-shaded head with displacement was done, where the skin-details where much better to discern than with sub-surface scattering.

Lastly the sound and timing of the expressions needed to be edited occasionally as well. On one shot Matthias demonstrated the problem in the plate with the body-double: The first part of the dialogue he wasn’t looking quite in the right direction, in the second part his face was obstructed when he delivered his line. Again, the right transitions needed to be animated to get the timing right.

And also there were shots where there was little animation in the face going on from the beginning, like in a reaction-shot of Clu just listening. It always seemed too static and the director wanted a maximum of still plausible movement in the face, despite the freeze-frame-like appeal of Jeff Bridge’s reference from the shoot. “Sometimes people just don’t move!” Matthias said, adding “Still, you can’t ever totally stop animating a face, especially if it should look photoreal. This would give it away instantly.”

“Sometimes… sometimes shots just clicked together from start to finish and went exactly as expected” Matthias concluded his lecture. There were not many questions and I tried to get to the “Raum Ulm” for the Adobe presentation as fast as possible. Unfortunately I still was trapped between two rather heavy guys and so I endet up as one among many waiting and pushing towards the room’s entrance. A few people squeezed themselves through the packed crowd from inside the room and the fmx hostess only let in as many people as had left. I was not among them but still waited in front of the door, eager to get in to Adobe’s D-SLR Workflow presentation. Then the presenter, Michael O’Neill, appeared from the back of the crows with a bowl of sweets. “Gee, you all wanna in?” — “Yeah!” the crowd replied. I have never seen so many Canon 5D’s and Rebel Ti’s in such close proximity. “Okay, then can we do another session during the lunch break? Yes?” The hostess shrugged. “Alright, then see you all in my break then!” and he slipped through the door. “Cool guy!” one girl uttered. And then the crowd slowly dispersed and I went across the Schloßplatz in the Holanka Bar, a bookstore-café to start this report on today’s lectures.

Upon leaving I had a little accident that resulted in a graze on my nasal bone that began emitting a small but steady current of blood. The fun part is that I wasn’t aware of it until half an hour later when people eyed me more suspiciously than usual.

Adobe-Schmoby

Adobe’s notebook,
originally uploaded by Phil Strahl.

I returned just in time to queue up once again in front of the Raum Ulm for an Adobe presentation titled Hollywood Visual Effects. Again masses of people were packed like sardines in a tin can and waited patiently in front of the door. But this time I picked a better place and was among the first that had been granted admittance by the hostess. As I walked past her I could feel the stirred crowd pushing into the room like a herd of kettle. I don’t know how but she managed to keep them back. Didn’t matter to me, I had a seat in the front row.

Michael O’Neill arrived after the Geisha VFX reel had finished and seemed a bit stressed but happy to see an eager crowd. A few minutes in his presentation he asked “Who has worked with Premiere?” Many hands were raised. “Who has worked with After Effects?” Almost all of them stayed up. “I love you!” Mike beamed enthusiastically. I bet if he would have asked “And how many have 100% legal versions of our products?” his smile would have vanished instantly.

What followed was the VFX reel of Shutter Island with rather basic comments on how to use masks and keying, and then he re-made a so-so crowd duplication in a shot of Gareth Edward’s Attila the Hun movie. A tremendous no-budget effort but with… how do I put this… obvious not-really-state-of-the-art VFX. So frankly I was a bit bored by the very basic VFX 101 which might have been totally new for some of the youngsters among the crowd. At least I got rewarded with some candy for making a smart-ass remark. Nobody likes smartasses like me, but what does Michael do? He encourages my behavior by feeding me candy! Still, I was hoping to learn something new. What really kinda blew me away was the VFX reel of Boardwalk Empire because as I saw those sequences I not once even considered them being so VFX laden when I saw them in the show.

And in that respect I was rewarded because the presentation turned out to be a clandestine marketing plug for bringing the Adobe CS 5.5 suite to our attention with some cool new features such as the “Refinement Plate” effect in After Effects (or something like that) which basically is the Roto-Brush as an effect. Further I really enjoyed seeing at least some of After Effects’ effects ready for 32-bit or at least 16-bit color depth.

Mike also presented the new Lens Blur which, according to his presentation, lets you set the focus-point anywhere in your live-action plate and have the rest blur physically correct. When it was Q&A-time I wanted to know how this worked, like After Effects triangulated the content and calculated a quick depth-pass. But, alas, Michael had a tendency to understand questions the wrong way (not only mine) and explained in detail to me how the lens correction profiles in Camera Raw worked. I knew this. He even boasted “We have made profiles for all lenses and cameras that there are!” I really forced myself not to talk back, because just a few days before I downloaded Adobe’s little lens profiling tool because they obviously missed quite a few lenses in their latest update. Still I wanted to know about the Lens Blur, so I pressed the issue: “Yeah, but that was not my question…” He cut me off and started explaining how a lens worked, what the aperture does and so on. Sure, he couldn’t know that I most certainly know better than him how that stuff worked, still I felt like he treated me like an idiot who had a Canon 5D set to full-auto. Frankly, I was pissed.

Still I remained at my seat although it was lunch break. Micheal had promised us the DSLR Workflow lecture and really stuck to his word, although it was obvious that he wanted to get through it as quickly as possible to enjoy at least 15 minutes of lunch time. Nothing really new here either, just a little Dynamic Linking here, a little Adobe Bride interaction and bouncing everything to Premiere or After Effects. I don’t know why so many people look down on Premiere because Michael demonstrated its USP extensively: The ability to work with native DSLR-footage (and even RED and Alexa raw footage!) in real-time without the hassles of converting everything to your intermediate codec as it is the case with Final Cut and Avid. Since Adobe got their hands on the RAW-color-specifications, even camera-raw for RED footage in Premiere and After Effects did not affect the speed and quality of the real-time playback thanks to their Mercury Playback Engine. That was the most impressive thing I’d seen in this year’s Adobe presentation. At least something.

VFX Politics

Jeff Okun presenting,
originally uploaded by Phil Strahl.

Today I had enough energy and motivation for trying something new and so I made my way across the street and into the other building for Jeffrey A. Okun’s²Politics of VFX. I was lucky and got a seat right in the front row so view and sound were excellent and allowed me to take some good photos of the speakers as well.

VES‘s executive director and FMX veteran Eric Roth introduced Jeff, chair of the VES, in very dear and complimenting words so the man got a big applause despite we, the audience, quite didn’t know why we applauded so enthusiastically. This blind and externally fueled enthusiasm should really give us to think. On the other hand, well, we didn’t applaud Hitler.

“Let’s talk about politics!” Jeff begun.

“I want everybody to raise their right hand and swear not to ruin my already rickety career because I am going to name names and expose people. Now do it and say ‘I swear’!” Yes, this was going to get juicy, I thought with a smile.

Jeff started a –god knows– overladen and ugly Power Point presentation. The first slide read “Visual Effects and Politics. Politics show up when you are: 1. Getting Hired” and I counted five different fonts, three different fade-ins and also three different colors. I was overwhelmed by this visual bravery. Jeff climbed the highest point of the stage, directly in front of the projection and lifted the veil on one myth: “You don’t get hired because of your skills. You get hired because people think you are an interesting person, like the way you are or because you have a good reputation. If you don’t really have a dynamic personality you won’t even get hired.” And Jeff went on telling anecdote after anecdote.

“How many of you are industry professionals? … Good. How many of you are students? … And how many of you don’t know why they’re here but thought this was a good way to spend some time?” I already liked Jeff, whose polarizing character and appearance could be compared to being the VFX industry’s Julian Assange.

“Everybody on a movie has an agenda. And creating the best result on screen is usually never makes it to the top-ten of their priorities. On top are promotions, family, friends, future plans, etc. And this can lead to the strangest and most wacky decisions during the making of a film”, like when the whole movie gets shot in a certain location just because the girlfriend of the director, who happens also to be the lead actress, wants to visit Malta. Or that, for example, the production designer hates the VFX team because it threatens his job and so he sabotages their efforts in as many ways as possible on set. Jeff also told how on the set of Last Samurai Tom Cruise (whose high-fives were vicious to the point that you would duck away to escape them) was ordered to be shot in front of a blue screen — in a navy blue suit. “This was not because they didn’t know any better. This was because of proving a point, that the VFX crew follows the orders of the director and not vice versa.” It sounded painful. “When you’re not Digital Domain or ILM you don’t have much to say on the set and are under constant pressure to just get what you need.”

“Cinefex always presents the same old making-of story over and over again: We met with the director, developed the look, made some tests, he loved it, we shot it and produced the VFX and everybody loved it… Raise your hands, who of you has ever been in such an ideal production where everything worked from start to finish?”. Silence. I thought back about past and also present productions I took part in which where tattered and torn by the interplay of individual agendas. And in the audience not a single hand was raised as far as I could tell.

The next set of slides with sickening word-by-word-transitions materialized on the projection. As we all waited for it to finally finish, Jeff added with a charming childish smile “I had so much fun doing that!” The slides essentially said that all politics come from a certain point of view and that “your brain sees what it is expecting, not what is there”, a statement underlined with photos of optical illusion graffiti like these in Toronto “by a group of people who just do this, they say, to mess with drunks”. That got a huge laugh. Jeff then continued showing blivets, impossible objects. “Trying to get the right perspective hurts your brain like watching a bad 3D-movie” he spoke, not missing to make a small remark about the stereo-conversion of Clash of the Titans. “A blivet is to perspective what reality is to your brain” was prominently placed across a slide in a stinging yellow color. “Can you really trust your brain?” Jeff asked rhetorically “No. That’s the point.”

“Some politics are good politics, some politics are bad politics and some politics are just crazy politics. These crazy politics get propagated by crazy people, and artists are oftentimes crazy people. And where do you have a bunch of artists together? In a movie production. So 90% of the politics you get are crazy politics.” Jeff deduced. Was it really that simple? Well, it made sense though. “The thing is, that politics affect the story. The Story… this is the altar before we bow.” Jeff paused for a moment. “And, by the way, the thing that killed 3D three times now is not bad technology, it’s bad stories. Now look at today: The big studios are dependent on flat stories about super heroes. Soon we will have super hero TV shows and commercials. The thing is, studios make movies for business, we do it because we love doing it.”

Jeff then talked extensively about the issues on Red Planet, how the story sucked, how the leading actor pissed everyone off and sabotaged the whole shoot so “that you were lucky to get two shots a day.” The VFX Producer quit, there was no VFX supervisor and Jeff only stayed on the set because he was promised work on the newest Harry Potter movie if Red Planet was thgouh. Things heated up further, so that the two remaining lead actors (the story had killed the others one by one before) filed restraining orders against each other and the only way to finish the movie was by shooting them one at a time and compositing them together. All that sounded dreadful! And this was just one of many productions that were going down the toilet during production. “You can tell how bad things are going when you look at the versions of a shot.” One had the version number of v1156.

“VFX is another word for not being able to say no”

The problem of all this is that the costs increase and the VFX department gets blamed for “getting crazy”. According to Jeff, the budgeting usually works like this: “Some 21-year-old one reads scripts all day at a studio and evaluates them, like The VFX on this movie will be $ 800,000! Then I read the script and say: That will cost you $ 25 million! Then the studio panics and asks us to cut costs. With some trickery we figure we can do it for $ 19 million which still is too much for the studio. Maybe you should order a re-write of the script? I suggest and then the rewritten version comes back with a new scene of an army marching though a city to the harbor and I say: Phew, with all that we are around $ 28 million and the studio will reply in lack of understanding: Why the heck is it now even more expensive?!”
On every film Jeff (and I for that matter) ever worked on, the final number of VFX shots went up, and so did the costs.

“What only matters is what ends up on the screen. Nobody will ask you whether everybody had a great time or you barely made it out alive.” If the movie is a success, so are you. If not, and you’re the VFX Supervisor or Producer, it’s your fault. “But always be aware of the agendas and politics of the people around you.” That was heavy. But hey, somebody needed to say it.

Pioneering for future legacies

After a few minutes’ break Eric Roth once again made the audience tingle with excitement as he was announcing another industry veteran, Bill Kroyer, who talked about the making of TRON from 1980 to 1982 in How Classical Animation boosted CG.

Bill Kroyer, then and now,
originally uploaded by Phil Strahl.

Bill started with a short history lesson and with what it was like when he begun at Walt Disney Animation and got trained under the tutelage of the Nine Old Men. In the 1970′s the animation studio was the same as it had been for decades, Disney was very reluctant (if not avoiding) to adopt new technology back in the day. To get this point across Bill threw up a slide reading “The only technology in the studio not available on Snow White…,” the picture below showed an old electric pencil sharpener. You couldn’t start much lower on technology than that.

Around the end of the 1970′s Steven Lisberger became intrigued with video games and wanted to make a film in that aesthetic of glowing colored lines. Out of that idea he initially envisioned TRON as a fully 2d-animated feature in mentioned aesthetic, but the project grew bigger and so did the ambitions of him becoming writer and director creating TRON. He recruited a couple of animators and flirted with state-of-the-art technology in bringing the vision to life: by modeling the world and the vehicles in the three-dimensional computer-space to create “Simulated Images” as it was called back then. This was where the training as a Disney animator really set them on course because at Disney “you always think spatially, you create the illusion of depth,” Bill explained as he showed a character-board of Jiminy Cricket, drawn from any angle with the underlying three-dimensional primitives sketched in. This board was over 70 years old!

As for the look they got design legend Syd Mead work on the “evil” architecture and vehicles (such was the famous light cycles) and French comic artist Jean Giraud, better known as Moebius, to work on the “good” designs like the Solar Sailer. Moebius also was one of the first artists to rely in his storyboards only on outlines and different shades of gray markers. When the storyboards were finished and the few artists on the project knew exactly what they wanted it to look like, the hard part begun. For that matter each story board panel had two numbers on the bottom, the first one indicating shot order, the second one defining the exact length in frames.

“A lot of graph paper”

The Light Cycle,
originally uploaded by Phil Strahl.

There was no unified software for creating computer graphics at the time, there weren’t even unified computers, “the companies even built their own computers.” Four of these companies were tasked to recreate the designs of Syd and Moebius in 3D space with combinations of primitives, in the case of the light cycle it was MAGI/Synthavision in New York, a company that was founded in 1966 by Dr. Philip Mittelman, “the guy who also invented the MRI,” Bill added.

“At that time there was software for creating and describing stuff in 3d, displaying stuff in 3d, even lighting stuff in 3d but nobody ever thought of software for animating stuff in 3d” Bill explained. So “animating” became a completely different endeavor than in traditional animation:

322,200 values,
originally uploaded by Phil Strahl.

First, the animators needed to specify how the models would move in the digital world. For example, what the turn radius of a light cycle was, how it would tilt according to what curve angle and all that — not approximately but exactly, in values and degrees. The same route needed to be taken for the scene geometry: Everything hat to be drawn and outlined in the right proportions with numbers and curves on graph-paper; yet it didn’t end there: Animations of vehicles and the camera needed to be put into numbers and the classical exposure sheet now had 7 columns for each object: XYZ position, XYZ rotation (yaw, pitch & roll) and frame number — for each frame³.

“That means 144 values for one second of animation” Bill’s slide read. The next continued with “TRON had 15 minutes of computer-generated animation. That’s 900 seconds of footage. With an average of two vehicles and a moving camera in each shot, the animation required 322,200 hand-recorded values!” Heavy.

But Wait!,
originally uploaded by Phil Strahl.

“But wait!” Bill continued like a TV salesman, “there was one computer on the Disney lot in 1980.” It was the computer that was controlling the intricate moving panes which were used for pans and other intricate effects in Disney animation movies. And this computer could even calculate and interpolate soft Bezièr-splines, something your average curve editor does without bragging nowadays. So Bill would sneak into the animation department with the computer during their down-times (usually at night) and enter the start and end values. With the tap of a button the machine would compute and display the inbetweens. “But it didn’t have a printer, so you needed to record the values per hand,” Bill reminisced with a smile. And the operators at MAGI took these exposure sheets with manually recorded values and their people, Chris Wedge among them, typed them up for use in their software. That’s how early computer animation was done.

Bill talks about the Chromatix 9000,
originally uploaded by Phil Strahl.

“The first time we would see any objects in motion was in 70mm projected on the stage,” another slide emphasized. Because creating, calculating, positioning, rendering and outputting the animation was such an enormous feat there were no second takes, everything needed to be dead-on the first time. “Occasionally you got the chance to tweak a bit. So we had to know and envision each shot exactly and completely before getting to work, something that has changed with increased computation power over the decades” Bill subtly nudged the audience verbally towards the “think before you animate!” concept that gets so often left behind.

Luckily for the small team, new technology was available to them during the production in the shape of the Chromatix 9000 computer with a big screen, that could calculate and display shots in a rough wireframe, although not in motion. The transfer from 3d-workstation to the Chromatix was in fact via a phone modem and one screen refresh would take about ten minutes. “People in the studio would just drop by to watch that, shouting with glee how amazing it was.”

Optical Compositing

For some it audience the cumbersome post-production process of TRON was new and even for those who knew, the recognition was palpable: For the crisp and digital look of the movie, the soundstage needed to be filmed with a large depth of field which, naturally, required a big amount of light, even more so for 70mm footage⁴. Each frame of the black-and-white footage then got transferred multiple times to 11″ by 17″ film and boards on which various portions got masked out manually in the end by Taiwanese artists with paintbrushes and ink. These single masks then were then used for various different exposures that got optically composed back together which meant an insane amount of film-transparencies, masks and compositing-layers. Not even the color glow was out-of-the-box, instead it required thousands of wedges to get the right amount of glow intensity for each character in each shot.

And then there was the problem of flickering blacks, its origin in the pipeline the team could not pin down for a while until the varying black-densities of the Kodalith lith-film could be identified as the source, as Bill explains in the video below.

“You should get the BluRay of the old TRON it has been restored beautifully,” Bill plugged the movie, adding with a serious expression “but it if hadn’t been preserved on film, it would have been gone.” Bill was talking in the last minutes of his presentation about what he coined Digital Nitrate.

Bill during Q&A,
originally uploaded by Phil Strahl.

I was sitting in the first row and took a photo of that slide as Bill looked at me and pointed at my D-SLR. “Did you know that photographers at the Academy Awards carry two cameras, one being digital and the other analog? Because currently we have no way of reliably preserving anything digital⁶.” The last words also appeared on his final slide, erupted into Keynote-fire and vanished to ashes.

Instead of a grim ending like this one, Bill hurried to a more uplifting finish, mainly directed at the students among the audience, many of them eager for a job as animator — the sooner, the better. “People hire always the artist and not the technician,” Bill told them, “Visualize and communicate, independent of your software to become a better artist. Skill in a classical sense transfers to new technologies, so no matter what tools you use, stay focused on honing your skills.”

Applause. Then Q&A followed, some of it in the video just below. And a fun fact: Rendering all the CG-sequences in TRON together took less rendering time than a single frame of TRON Legacy. Amused chuckles in the audience.

Bye-bye

Unfortunately that was day two for me. I didn’t stay for the VFX of “The King’s Speech” and had by god no intentions for another round of Adobe’s presentations. In fact, my head was as tilt as any good pinball machine after a hearty bump so I left with the would on my nose for a bite to eat, a dose of caffeine and a smooth ride to the hotel.

The first row in many rooms is now reserved for lecturers so I had to abandon one of my favorite seating places of past years. Further the exit of the main auditorium has been moved to its side, probably because people pushing their way out and people pushing their way in made the access to popular lectures a pain — literally. Further, the FMX had matured and grown and hence offered even more parallel lectures in the opposing building.

Shortly before ten I had my ticket, had set off my first tweet, got my welcome bag, had my ballpen at hand and took a seat in the second row: I was ready for this year’s conference! Since I was a bit early I came to see the last of the screened animations of the Filmakademie Baden-Württemberg, a satirical take on the juicy topic of Austrofashism in the 1930′s.

Going Mobile

Neil Trevett,
originally uploaded by Phil Strahl.

Then they rolled this year’s FMX trailer, in my opinion a fresh idea after last year’s rather featureless animation. Then the first speaker of the event got introduced, Neil Trevett from nVidia, incidentally nVidia being the event’s main sponsor. I was preparing myself for having some marketing mumbo-jumbo dropped onto me but it wasn’t that bad after all in his lecture Movie Making and More, All in the Palm of Your Hand.

First he lined out how proud nVidia is of their Quadro series, CUDA and that all of this year’s Oscar nominated VFX films used nVidia technology somewhere in the course of the production. Big whoop — Adobe can claim the same thing probably.

Then followed some more nVidia-green Power Point slides depicting how the market of the “classic”, stationary desktop computer was more or less saturated today, even the growth for notebooks and netbooks started to level out, yet the market for mobile devices such as smartphones and tablet-computers exploded exponentially, with the Android operating system already ahead of Apple’s iOS. And since nVidia wants to get a big slice of that cake as well they started development of the Tegra series, high-performance graphics processors with as little power consumption as possible.

But that’s not enough: The old x86 chip architecture is also in decline since tablets and smartphones don’t come with a constant connection to a power grid, so the old ARM architecture will become a big thing of the near future. For nVidia this is Project Denver, ARM with integrated GPUs, while Microsoft’s Windows 8 will also support this kind of hardware. nVidia also works together with Adobe to make the execution of Flash and AIR applications more economic on mobile devices. “Apart from playing back video, the main task of Flash seems to be blending and filtering. And we let the GPU do this.”

nVidia seems to be pretty tight with Motorola as well, because their cutting-edge processors chips are and will be inside cutting-edge Motorola hardware as a chart showed. And I realized that I will carry a laptop with me for a couple of years longer, because what I want and need to do on the go can’t be feasibly done on a tablet anytime soon. But for the average person who wants to listen to music, surf the web, read eBooks and write emails a tablet with a full-sized keyboard/dockingstation/battery-combo (like the ASUS Transformer) might be just what they need. USB-input-device support anyone? Ice Cream Sandwich?

In the very near future mobile devices will have quad-core processors hence multitasking should become as common as necessary, further the portable devices will provide autostereoscopic displays, such as the Nintendo 3DS. Last but not least the resolution with increase drastically to an estimated 2560 by 1600 pixels (nVidia boasts it “Extreme HD”) which effectively results in a resolution of 300 dpi on a 10-inch tablet — print quality.

But what about power consumption? “As paradox as it may sound but more cores save you more power: The excessive cores only turn on when an application needs them” Neil explained.

Gee thanks, Neil, that’s pretty interesting, consumer-wise, but what about our industry? “Tablets in the movie industry can be and currently are used in three different ways,” he continued. One possibility is to port software to mobile devices. I snorted mentally: Anybody who tried the incredibly limited version of Photoshop for Android knows that a smartphone is still way too underpowered to get real work done. “By 2014 mobile devices will have 100 times the performance of today.” So will CUDA also work on phones? “No, CUDA’s power consumption is one of the biggest stepping stones in that respect” Neil shrugged. But that didn’t matter to me, because “the next generation of tablets and smartphones will be aimed at the demands of artists and the creative, like by additionally offering stylus operability and pressure sensitivity. The first devices will probably be released this year.” I dumbly smiled at the poor person sitting to my right. I am so buying into this!

The next useful integration of tablets was wireless tethering with desktop applications, because “this ‘new’ way of interaction with a user interface that is so much more intuitive than having to nudge a mouse pointer across the screen.” Adobe released for developers the Adobe Photoshop Touch SKD which opens developers the door to think of clever ways to integrate a tethered device. For example, it could serve as a color palette where swiping blends colors together. The resulting colors then can be returned to the host application. EDIT: Adobe recently me an announcement via eMail to update Photoshop and get complement software from the AppStore. Pity I roll with Android.

The third possibility is using tablets as cloud-clients. nVidia has already a Flash-based technology for combining collaboration with Tegra-powered tablets called “Studiopass”: It lets you upload, stream, annotate and comment in real time on video files with others. It is also possible to use the tablet as intuitive virtual viewfinder that’s connected to a render farm that returns within a few seconds the rendered image to the device. And, surprise, Studiopass is built on Flash.

Just when Neil got glowing eyes and popped up a slide in purest nVidia’s corporat0- design reading “Super” Computing, the time was up. Close call!

Baking Light

François takes a pic,
originally uploaded by Phil Strahl.

I was too lazy to cross the street for Approaching a CG Production so I stayed in the König-Karl-Halle for Bakery Relight by Thomas Driemeyer, Emmanuel Turquin and François Belot from The Bakery who talked about their company and their software, Relight. I was a bit distracted by the amount of people swiping away on their iPhones, Joseph Olin among them, while François outlined the history of the company in southern France. I only got to hear the last few words, “Sand, Beaches and Girls!”, although what I understood as “Beaches” could have been some other word. Or maybe it just was the thick French accent and the tendency to pronounce some words French, or switch to French vocabularies entirely.

Relight is essentially a tool for lighting and shading shots and to develop looks, realistic as well as artistic. “This tool was designed by artists and not by engineers” they noted and indeed: As Emmanuel showed a quick overview of the interface it seemed quite straightforward, a bit like Katana. “You don’t need to make a break every time you want to render a change on your way to the final shot, you always work with the final and that really fast.” Even little comfort features are implemented, like solo-ing certain light sources.

It plugs directly into Maya, 3dsmax and XSI, it comes with its own hair and fur system that is created at render-time by its production renderer that produces really good and clean images. The renderer itself is a Reyes-like rendering algorithm, iterative and optimized for fast feedback in production quality. But this need for heavy-duty-caching takes its toll on memory consumption I guess.

Once the scene’s visible geometry is cached by renderer (many millions of polygons take a couple of minutes) Relight unfolds its performance. Lighting and rendering a forest-scene with hundreds of trees and atmospherics only took a minute on an average notebook computer — impressive. And once the geometry and the first lighting pass is cached, the renderer gets even faster with every change. It keeps track of what has been touched or adjusted and only updates the dependencies, what had been affected by the change; and nothing more. “Depending on how many processors you use, the suite scales pretty well accordingly.” Even motion-blur and DOF (depth of field) are fast.

Another big thing of Relight are point clouds for quickly carrying out calculations that would bring a raytracer too its knees, memory wise. Point clouds (or disk-clouds to be more precise) are independent from the underlying geometry and make it possible, for example, to render ambient occlusion in fur stunningly fast. But ambient occlusion is only one application. Point clouds can be used for glossy reflections, environment light (from HDRIs, for example), area-lights or sub-surface-scattering.

Towards the end, François added that since Relight provides feedback so fast, it is a great tool for budding lighting artists and there are many collaborations with schools. Then he closed with a sentence in thick French accent that sounded to me like “Thank you very much for your invention!”. You’re welcome!

Arnold, the Tracer

Whereas The Bakery‘s Relight tries as much as Pixar’s RenderMan to cheat around raytracing, Sony Pictures Imageworks has a completely opposite approach with their proprietary renderer Arnold. The lecture by Marcos Fajandro and Larry Gritz Path Tracing and Unbiased Rendering started off by comparing rasterizer/Reyes renderers and ray-tracing.

Of course, Reyes depends on shadow maps and offers no ray-tracing but can be insanely fast, whereas a pure ray-tracer craps out completely with too complex geometry (see the issues with Davy Jones as discussed on the FMX/08) and/or/like hair.

So what do you do? Integrate some limited raytracing into a Reyes renderer or hack rasterizing & (deep)shadow functionality into a raytracer? Both options aren’t exactly irresistible. So the folks at the Spanish company Solid Angle, I haven’t heard of ever before, developed Arnold.

The first feature film rendered with Arnold was the, in my opinion terrible, Monster House and now since a couple of years it has become Sony Pictures Imagework’s only renderer.

Arnold is based physically on the Monte-Carlo methods, it uses no storage for lighting and global illumination information, has only one quality knob to tweak on (namely the sampling) and outputs the final in a single pass. Yes, Arnold is a ray-tracer par excellence and traces millions of rays. “And when you trace so many rays you need to make the rays really fast”. Further, Arnold sports networked, programmable shaders, subdivision surfaces and yet can handle “hundreds of millions of triangles and hair splines” and virtually “hundreds of gigabytes of texture-maps”. Impressive!

Since everything (everything!) is ray-traced, the final image is optically seamless. There’s no need for say, a shadow pass or motion vectors. My worst fear of such a renderer is that it probably takes ages just to get a feedback of lighting changes, but that is not the case: An example video showed a car in Maya in the viewport whose camera was turned around it. As soon as the operator let it go, Arnold kicked it and its interactive mode rendered the selected region in big blocks, then smaller blocks and even smaller blocks until you could make out the raytraced details of car paint and reflections. This refinement process took not more that two seconds.

“In the end what matters is not how long you wait for your beauty-rendering but how long you wait for feedback of each iteration in the (re-)lighting of a shot. Since an artists costs you approximately $ 40 an hour, this way of saving time saves you a big amount of money and the artist downtime between iterations” Marcos explained. I heard several mental kaching-sounds in the audience.

Making raytraced motion-blur efficient is also important for the final rendering. A fully lit and shaded scene with production-quality deformation motion-blur rendered only 15% longer than without. Arnold even is proficient when it comes to volume renderings.

“The joy of Arnold is, that everything just clicks together seamlessly when you raytrace everything in your image. If you use different renderes for different tasks, even simple things like shadows can be a problem, like shadows in volumes. With Arnold there’s no catch — you get the whole package in one go with ambient occlusion, motion blur, soft shadows on motion blur, caustics, etc. ” Marcos summed up.

And just as joyful is instancing of geometry that gets loaded into the memory once for whatever number of instances. One shot of Alice in Wonderland had 61 million triangles and could be rendered in a single thread’s 15 hrs which boils down to an hour or less in a farm.

“Still there is lots of room for future improvements” Marcos concluded before Larry took over. Larry talked in more detail about Sony’s way of VFX and relighting using the upcoming Smurfs movie as an example.

For every shot with CG-integration, they shot a HDRI of the set with an incredibly expensive Spheron camera. In fact, not just one panorama, but two in different heights so with the set survey data it is possible to calculate and to recreate the scene geometry via triangulation roughly in CG. Then, for quality reasons, the light sources and instruments in the panorama get replaced by Arnold lights and erased from the HDRI panorama via Katana. The actual plate from the camera is projected onto the low-poly scene geometry, then the HDRI panoramas get also projected into the scene. “It’s kinda the same what was done for Benjamin Button” Larry added. Now the CG-characters are imported into the scene and get their lighting and bounces from the surfaces that the plate and panorama was projected onto.

As mentioned above, Arnold has an open shading language. “Traditionally, shaders are black boxes to the renderer that just return some values to it; their units are sloppy oftentimes and their underlying C/C++ code can also crash the renderer altogether. With an open shading language, we don’t have these issues.” Instead of returning color values to the renderer, the Arnold shaders compute so called “closures”, descriptions of how the surface will react to light, and pass these numbers on to the renderer, which can decide what to do with them. “This is also 20% faster!” Larry added happily.

What working with these “closures” exactly meant and what it had to do with ray-budgeting and Multi-Importance-Sampling (MIS) for the Monte-Carlo raytracing I did not fully grasp but Larry had some links (and his email address) on the subject, in case anyone was interested. I reproduce them here for the same purpose:

• http://bit.ly/s10shaders
• http://opensource.imageworks.com
• lg AT imageworks.com

Some features of Arnold like sub-surface-scattering are performed with point clouds on-the-fly, although the renderer still employs Monte-Carlo methods when point clouds won’t suffice. Since Arnold is proprietary, Imageworks developed a plug-in for XSI themselves and a very basic one for Maya.

But why did Sony Pictures Imageworks settle for just Arnold a few years back? “The pass management grew so complex and bloated, it was hacks upon hacks, that overwhelmed the mental capacity of the TDs. Now it’s back to just having a single pass. Lighting got so much faster and in the end. Lighting and rendering a single pass still is faster than rendering lots of different passes and trying to get them working together.” Larry explained, and Marcos added: “When you reduce time somewhere in the process, however, somewhere somebody uses the new freedom to add more complexity.”

Currently Arnold uses only the CPU despite the trend of making everything CUDA- and hence GPU-compatible. “Porting would be complex and time consuming and in the end wouldn’t speed up things significantly, so we spend the efforts to optimize the CPU code instead. Further, the terabytes of textures would need to be streamed to the GPUs as well…” Larry summed up the bottleneck situation.

“We still have a couple of minutes left so I’m gonna show you the Green Lantern trailer. Everything was rendered with Arnold.”

Fun fact: The plate at the end of the trailer with the title and release date also had a rather small line reading “Also playing in 2D theaters.” Like it or not: Stereo has finally arrived and it is here to stay. Deal with it!

And here’s a LaTeX-set presentation on Importance Sampling for Monte Carlo Ray Tracing from 2006 with lots of pretty pictures (and some equations).

Coffee!

Haus der Wirtschaft,
originally uploaded by Phil Strahl.

The few hours I was asleep took their toll and I was wasted for the break. I schlepped my bag and camera to a long queue in front of Starbucks where I got me a motivational Caramel Macchiato so I had the energy to look for food. I found an Asian noodles stand in a subway station and made my way back to the Haus der Wirtschaft, munching on fresh cooked vegetables. The remaining ten minutes I had an espresso in the showroom of the FMX. When I felt ready for some more lectures and headed into the König-Karl-Halle and was surprised to see all the good seats taken. So I settled for a suboptimal seat next to a German student who was constantly eating or drinking something and merrily ignoring the ban on recording devices. Funny thing though was that he thought I didn’t speak German and I was in no mood to shatter his belief. So I sat there with Ophelia, my notebook, in her bag on my lap and waited eagerly for the next lecture to take place.

Render de Janeiro

Building, Lighting & Rendering “Rio” by Blue Sky’s Andrew Beddini was up next. Andrew somehow reminded me of a friendly Ben Stiller character and his engaging lecture was a pleasure to listen to without once the need to close my eyes for just a couple of hours. Or maybe it was because I was so close sitting to the pumped-up loudspeakers that I feared my ears would pop as they rolled the FMX-trailer.

Blue Sky, founded in 1987, is a veteran of the industry, they even took part in creating CG sequences in Disney’s original TRON. The next milestone was the animated short Bunny in 1998 I remember seeing at the Ars Electronica that time. I even remember that it was the first animation to employ the nowadays obscurely named Radiosity.

People really like to talk about their renderers I realized, as Andrew started summarizing the features of their proprietary renderer with the featureless name CGI Studio, or just Studio. On a rendering from 1993 Andy demonstrated the features their raytracer could produce back then which they still use today. Some of its features used for Rio were secondary rays, true radiosity, raytraced soft shadows, tesselation of beziér patches, a procedural shading pipeline and the possibility to use procedures for much more than just shaders. Rio is a feature of 1800 shots, and 80% of it had to be done in just 5 months so the focus was on creating a pipeline that was fast and efficient.

Character lighting was one of the main and foremost concerns of the production. And “good lighting needs good development.” The character of Susan with fair skin, glasses and big eyes was the first testing ground for plausible shading and lighting. First they applied the sub-surface-scattering-shader to her skin and realized, oh boy, that it looked not really convincing. So the first thing was to implement a density adjustment into it, then light transmittance through the skin, radiosity and a subtle but necessary secondary transmittance model that made the skin softer still.

Eyes behind glasses were another challenge because “eyes are critical for emotion. You really want to get the eyes right first. If they are off, if the audience doesn’t buy it, then all the other efforts are in vain, so get the eyes right!”. First they raytraced the eyes and the glasses but the physically realistic look did not work with the stylized character design. For example, the shadowing of the glasses under the rim was just too dark with the realistic refractive index of 1.5, only 1.1 was just about right. Can you do this? Should you do this? Andrew made it heard: “Break reality to make it work for you!” What was behind the glasses was rendered separately with a plate of the background to yield realistic refractions of it, as for the reflections, the reverse angle shot was used ever so subtly. But still the eyes didn’t look alive, so Blue Sky went the whole nine yards and rendered for the eyes a UV pass, a reflection pass, an object pass so the highlight could be adjusted in Nuke accordingly. This finally gave the eyes that certain something that was missing and was tweakable in every shot.

The set creation of the favelas was a difficult task as well. After the first color studies the assets were created with low complexity in a modular fashion and could be combined to a very organic whole according to a plan of the set’s final layout. “If you want to sell a set to your director render it in gray with just the ambient occlusion and he’ll love it!” Andrew joked. “There were absolutely no texture maps involved, everything was done procedurally with using the world space coordinates. Also we don’t need a level-of-detail (LOD) system.”

With just one building block, the lighting department instanced the geometry for lighting and mood tests of what the location might look in broad daylight, in the afternoon, at night and with atmospheric effects.

Raytracing and motion-blur — another dreaded combination of mine. But to tackle that problem Blue Sky employed a lot of tricks that made their lives easier. One of those was modulating the render-resolution accordingly via script: “At some point you can’t tell whether a 2k image was blurred or a 1k image was blurred, so if the renderer dials down the resolution you effectively are four times faster.” Another trick was using Nuke’s excellent vector blur: The camera and a Z-Depth channel from Maya got imported into Nuke and was blurred there in post.

A slide depicting the favela at night appeard on the screen. “In this particular scene we had about ten thousand light sources” Andy spoke and paused for emphasis. Ten thousand light sources with raytraced shadows?! “The thing is,” he continued “we use the same pool for all shadows, they get calculated at the same time. So if we’re having one light source or ten thousand only adds 10 to 15 % to the overall render time.”

Since the protagonists in Rio are birds, the sky is an important part of the film as well. “About 30% of the picture is sky!”. So how do you get a beautiful art-directed sky? The traditional answer is to have talented matte-painters. “The problem is, that we had four to five matte painters but hundreds of skies to paint. So this wasn’t feasible. Further the feature is in stereo, so…” he switched to the next slide depicting various clouds on a black background “…we had our art director paint a style guide for clouds we then built in 3D with volumes. We rendered each cloud with surface normals from 38 different camera tilts so you had almost any angle. Then we imported those onto planes in Nuke and populated the sky in the 3d-space there with the clouds.” The normals made it possible to vertex-light the cloud planes in Nuke. “This is really fast and you can churn out skies quickly. We did this for an entire sequence at once, not only for a single shot.”
But sometimes you need hero clouds, especially when they needed to have a volume and depth or some advanced lighting effects, like transmission on the edges of a key light. These clouds got individually modeled and rendered. The atmosphere-gradient with the sun was also done as a dome in Nuke and when you throw everything together — voilá — there’s your final sky! This sky then got rendered in Nuke and was used on a plane in Maya for the reflections on the water.

Generating a vista of Rio de Janeiro, especially a stylized version that still looks as photoreal as possible was the next challenge on Rio. The foundation for the environment was survey data of the topography that got artistically adjusted with some liberties. Still, it was built to scale so the renderer would deliver correct results of atmospheric effects, for example.

The vegetation was, like almost everything in Rio, procedural with implicit surfaces. But here the procedural approach posed some difficulties: it is hard getting a procedure to a single point. As for the shading, two world space procedural textures were created, one for the granite and one for the plants. A simple rule that depends on the steepness of the topology then blended between the two, so vertical walls would have no vegetation and soft slopes would be fully covered in trees. Finally, the city’s buildings were roughly modeled, but also were designed for procedurally driven variety: They could have differently spaced windows, doors, floor-segmentation, surfaces and rooftop structures.

What surprised me the most was the way the lighting was defined and carried out: It was script-driven and those scripts were simple text-files the production edited with NEdit in Perl and Python. “This is great because the files are small and portable, you can send them securely via email, and you don’t need to open Maya just to twist a light a bit to one side.” Blue Sky even has an interactive front-end called “Quick Render” to render models and light situations without waiting for Maya to finally launch. And there was more: “Light sources are not defined in the world-space but in camera-space,” instead of XYZ they are described by values for rotation, elevation and distance from the camera. Again, you don’t need Maya to adjust a light; creating a consistent rim-light can be done with a few lines in a text-editor easily.

Finally, even the atmospherics were rendered in favor of a Z-pass. “A Z-pass always has artifacts along the edges so you need to render it really big.” And once you have time for that and to add fog & haze in post, you can render physically much more plausible (and within the scene reflected) atmospherics.

A Hairy Subject

We only had a few minutes to let sink in what was heard before Mohit Kallianpur from Disney Animation Studios continued with his lecture of Untangling “Tangled” where he was the Look & Lighting supervisor and described the history of the look of the movie.

In 2007 the movie had a much darker, browner tone to its concept art until John Lasseter intervened and pointed the production in a more colorful and saturated direction. Still, that was relatively late in the pre-production stage and there was not that much time left to get the movie done. So Mohit got down to the root of art direction and formulated three principles: Stylized shapes, illustrative colors and believable textures.

Shapes

Then the research begun by watching and analyzing the shapes, colors and appeal of the old Disney classics Snowhite, Pinocchio and Cinderella. Especially the latter had a certain shape language of flowing curves visible in almost every shot, a graceful harmony. Moreover, a set of signature shapes of that movie was collected consisting of various bell-shapes, s-curves and leafs. Everything in the film would follow these shapes, even the canopy of the trees.

Based on a very impressionist and rough mood-painting that followed the shape-guide, the team produced a full CG version of it as reference. This made it obvious that the language of shapes worked well, but the painterly appeal of the surfaces was too stylized. So the world needed to have believable textures, not rely on impressionist suggestions of detail.

Architecture

The architecture of Tangled was influenced by European cities, Disneyland (!) and Pinocchio: Everything should be small, friendly and approachable. The buildings are not tall and they flare out in a curve, they appear even chunky and beefy with no sharp corners and a very organic and hand-built feel. And like this Mohit wanted the architecture on Tangled to look: old and used but not decrepit or dirty.

Color

But what does “illustrative color” now mean exactly? The production settled for a lush saturated palette and always a play of warm against cool: If the light is cool, then the shadows should be warm and vice versa.

Legally Blonde

Yet the most intricate and most critical task was to get the look of Rapunzel’s hair perfect. Much like the sky in Rio, the hair in Tangled was a character by itself. The first tests showed that the hair shaders the studio had developed so far didn’t do justice to blonde hair. At all. So research needed to be done and a hair model was found and her hair photographed in probably any appearance (even wet) and every possible lighting situation. Also commercials of hair care products served as a valuable reference “since they propagate the ideal on their packaging we wanted to recreate in the film.” Even a PhD student did extensive research on the subject and in the end a feasible shader was programmed:

First of all, hair comes in individual strands and strands that make up a volume of hair. Moreover, these strands have a top, a bottom and a body with an uneven surface, none of which is like the cylinder-representation of hair we all used to work with. So hair has a specular reflection, so far so good. And for black hair that’s usually all you see. Light hair also has a sub-specular portion that is a broader highlight in the color of the hair, whereas the specular highlight has the color of the light source. Then hair has a transmission value when lit from behind, and multiple scattering is important for light colored hair. Still that’s not enough because a hair rarely comes singly: There’s also a volume diffuse portion, that is backward scattering of the light and volume transmission, which scatters away from the light source through the volume.

Now if you add up those five components it looks good but not perfect, a certain richness is missing. This “richness” comes from the light bounced back onto the hair and ambient occlusion. Then, and only then, you are rewarded with beautiful blonde hair. But for eyebrows, fur and eyelashes you still need a different shader because the blonde shader needs a volume to work with after all.

Appeal

And another question: What constitutes appeal? What makes a person look appealing in a movie? How to light a character to look appealing? Mohit dug deep once again and looked through lots and lots of glamor-shots of actresses and actors of Hollywood’s golden age in search for unifying principles. And his answer is simple: “Cheat!”

The starlets on photos (and in movies) were always lit by a soft light, no matter what the rest of their environment looked like. Then there needed to be hue in the shadows, the dreaded “graying” of multiplying occlusion with diffuse passes makes characters look sickly. Since the eyes are the window to the soul, they needed special attention and always a specular reflection, no matter what. And as painters suggested, the cavities of mouth and nostrils should not go completely black but into a warm darkness. And unappealing colors (usually green bounces of leaves and grass) also make a character look weak and sick, so they had to cheat there as well. Finally a subtle bloom on the highlights never hurt anyone.

Look

On Tangled there were 19 look development artists that in the end produced over a thousand paintings of looks. And lighting was important to elevate the mood that was already there by narration, staging and framing. As Mohit showed some examples of lighting I could not help but remember the words of my mentor, Fraser Maclean as Mohit uttered them as well: “Put light where you need it, not where it realistically would be.” Indeed, the progression of shots he showed were lacking any continuity of the origin of the lights. “And here… well I don’t know where the light is supposed to be coming from, there is no window there” …but still it works perfectly in each shot. It went so far, that even the murals in the tower could be toned down or changed in opacity on a shot-by-shot basis. “Light shapes add to the drama.”

And so does saturation. In dramatic sequences the saturation got sucked out of the pictures, only to return almost fully later on.

Art-Directed Trees

Once you can art-direct everything, you have to art-direct everything. The R&D department provided a simple tool for Maya that allowed the modeling artist to draw a couple of curves and the plug-in would make a tree with branches out of it. Moreover, they also had a tool that would grow leaves into a pre-defined canopy-shape which was really fun to watch.
And since there are so many trees in a forest, the models were also switched to “brickmaps”, a RenderMan-term for low-res voxel-representations of a model, if far enough away and automated “stochastic pruning”, which meant that depending on the distance to the camera, not visible leaves would be automatically switched with low-poly models or removed entirely.

“In the end we generated too many trees and had to hide them in haze or atmosphere in post” Mohit admitted. That really was new to me, having accidentally too many in the final picture of something so complex such as trees!

Tangled was rendered with RenderMan (finally another Reyes renderer today, I was almost worried!), had 1380 shots and 55 lighting artists which, in sum, resulted in 9.01 million hours of a single lighting thread. “That’s more than 1028 years”. Ah, statistics. It’s like I tell you now that this blog post is already some 6000-odd words long.

Physically Based Shading

Ben Snow,
originally uploaded by Phil Strahl.

I used the short break to get back close to the front in row 2 and to empty my can of Starbucks “Doubleshot Espresso” to be all up an ready for Ben Snow’s and Christophe Héry’s Physically Based Shading at ILM¹.

Ben showed some definitions at first outlining GI, IBL and HDRI. “We’re not trying to reproduce reality,” Ben made clear, “but filmed reality” and described a little ILM’s history in their attempts to achieve this goal over the years.

The practice of trying to capture as much information from the set grew over the years in scope and professionalism as well. In the 1990′s they started filming and photographing light probes on set, 18% gray balls, to recreate the light later in the computer. But soon they realized that was not enough. Along came six photos in each direction for cube-maps and then the familiar chrome ball, a technique I never quite got to work for my own projects. Indeed, the chrome sphere’s reflections were rather low-res and you needed to paint out the photographer every time as well.
So in the “early days” ILM made heavy use of texture maps with painted in highlights and shadows, for shading they employed the Cook-Torrance specular model. Their light rigs were also pretty basic but could handle real world situations already rather well. Occasionally shadows grew really dark and some ways to cheat were e.g. spot lights or churning down the overall shadow opacity².

Then came Perl Harbor when they really needed to crank their depicted reality up a notch since ambient occlusion wasn’t enough. So they developed what they called Ambient Environment Lighting, which essentially is creating a pass for the ambient lighting via ambient occlusion: Rays are cast in a hemisphere around the surface normals, then a number of rays hitting other surfaces dictates the occlusion; and a pre-pass is done to calculate the average direction of the light. At least that’s what I copied from Ben’s slides. The film was a milestone nevertheless because Michael Bay, the director, was not able to tell the difference between CG and live-action-footage anymore.

What happened after Perl Harbor was boosting the quality consistently further. 8-bit images had served their purpose well (including in Perl Harbor!), but the need and time asked for floating point precision. Also, no mirrored balls would be photographed anymore in favor of photographed panoramas.

So a couple of years afterward came Iron Man and asked for the realistic depiction of, well, iron and metal that needed to match the practical suits on set. And a thing that had been troubling the folks at ILM was anisotropic highlights that appear on brushed metal.

Further, a new approach for taking HDRI panoramas was employed and meant taking a series of photographs from a tripod in all directions, although (as it appeared to me) not in a truly high dynamic range but only covering two two exposure brackets — too little from my experience but still it worked for them.

New Frontiers

And just as they thought they had mastered metal, along came Terminator: Salvation where it was impossible to cheat any longer and they had to move to a new paradigm for lighting and shading in RenderMan. ILM’s goal was to get a simpler, more intuitive and physically based system of lighting and rendering:

The quality of shading and lighting requred a BRDF-model that not only looked right but also acted physically correct in terms of energy conservation. In short this means, that the rougher a material is, the weaker is highlight gets, otherwise the energy (= light) reflected would be bigger than the light received — impossible³.

That also meant a normalized specular highlights with a more physically plausible specular falloff, so the further away a light source is, the weaker the specular highlight gets, depending on the roughness or the “Normalized Importance Falloff”: The intensity of the highlight falls off on rougher surfaces. For tight specular highlights like chrome or mirrors the light source has to get a long way away before dimming. The broader the speculars are, the more quickly they will dim. For this to work the light needs to have a physical size in the system, so no more point-lights or directional lights at ILM

“This was hard for everyone to adjust to and Christophe and I remember some very passionately fought holy wars” Ben remembered.

On the set of Terminator: Salvation the chrome spheres were back, but differently. Now they were moved and shot in motion so their reflection could be used on moving models. “We still don’t have a great way of capturing HDR moving images,” Ben continued in front of a turntable of a T-800 in front of a steelworks plate, “instead we shoot HDRIs with stable lighting and we also shoot our chrome spheres so we still get our FX, strobes, sparks, etc. And on top of that we applied some pyro elements shot on film and used them as reflections or area-lights in the scene.”

Iron Man 2

“We now had our tools that worked, they weren’t really mature but technically they were robust” Ben noted. For image based lighting ILM then used a graphical tool, the Environments Browser, to quickly define light sources within the image so they could be recreated with area lights. The match-move of a shot creates the environment and the HDRI panoramas get projected onto the geometry and you basically end up with a HDRI-mapped recreation of the set environment in 3d. That enabled the artists to render dynamic HDRIs from any position in the set to be used in image based lighting of the digital characters.

IBL via HDRI

But Image Based Lighting also poses some risks or at least things to watch out for. An important thing to be aware of is that IBL-lights are treated as infinitely away point-lights from the scene. A fact, that comes distractingly into play when CG-lights with a discrete position within the scene are meant to work alongside an IBL dome. Then it really takes an experienced photographer and visual effects team to produce good and usable HDRIs from the set which is essential for this approach. Last but not least the pipeline should support the the floating-point nature of HDRIs or else one might run the risk of losing dynamic range along the process when editing HDRIs.

On set ILM still photographs chrome and grey spheres but only as references. This only works properly if the film crew is in the habit as well and does not treat those shots lightly or worse, forgets about them. Since the VFX team later needs to match the lighting of the shot everybody liked on set, those references should be shot immediately after the director yelled “cut!”. “You wanna make sure that your spheres are as big as possible in the frame and you wanna make sure they are in the right spot. And you don’t wanna shadow or be reflected in the sphere” Ben reminded. Sometimes when the spheres are moved like an object though the scene, it is advisable to take some static sphere footage as well.

“On the set as soon as they say We’ve got it! then off you go and get the references.” These references, of course, should only be shot for shots that will have CG portions but if there is ever any doubt that a certain shot definitely will not require CG, one should still capture the references to be on the safe side.

“And be serious about it. If you don’t take this seriously nobody else on the crew will” Ben shared his experience with the audience and switched to his last presentation slide, titled “How we capture HDRIs” which I shall reproduce here:

• Canon 1Ds Mk3 with Sigma 8mm fisheye lens
• Nodal Ninja & Tripod
• Remote shutter trigger
• 0.6 ND (2-stop) filter (depending on how bright it is)
• 7 exposures, 3 stops apart
• Direct sun f/16, ISO 100, center exposure 1/32 sec

After a quick slide of acknowledgements Ben handed over to Christophe who would once again talk in a little more detail about the math-laden background behind the presented concepts.

Since my senior-high maths-teacher sucked out all the fun I ever had with mathematics, I wasn’t in the mood of trying to follow Christophe’s every word but I got the general idea:

Calculating reflections and IBL with the Monte Carlo approach requires a high amount of samples for a clean picture. Since the distribution of the rays is random, you end up calculating a lot of stuff you will not really see in your final rendering. So he came up with MIS, Multi Importance Sampling, that takes into account what the camera sees, that the light illuminates and identifies an area in the lighting-dome where rays have a probability to affect the final rendering, such bright portions in the HDRI used for IBL. With the same number of samples used for rendering, which are only weighted differently, in the end compose a much more pleasing result because you only calculate what you need.

To implement this into the renderer, the BRDFs and lights need to provide eval() and sample() methods, where eval() returns a color and a pdf for a given input direction and sample() that returns an array of directions and pdfs: For instance, in a dome IBL situation, these will be the vectors to the bright spots in the image.
I just copied this from a slide of Christophe’s presentation, so don’t ask me what it all means. The only thing I am pretty sure is that pdf in this context stands for probability distribution function and not for Adobe’s favorite way of storing their user guides.

ILM’s BRDF specification has to follow a number of principles too: The shaders now must be normalized e.g. energy conserving, they must “substitute” to ILM’s trusted but old Cook-Torrance model, they should be anisotropy aware and, as always, efficiently computed.

Their solution now is D-BRDF based on an yet unpublished paper by Michael Ashikhmin and Simon Premoze; the Beckmann distribution⁴ and no masking: the reciprocity term is simply 4.0 * V.H * max(L.N, V.N).

For the nerds among you I even noted the links Christophe’s presentation closed with:

• Eric Veach’s paper⁵
• Last year’s SIGGRAPH course
• D-BRDF

You’re welcome. Now where is my mind?

After this heavy information laden day I skipped nVidia’s panel discussion and went straight home to type up this blog post. who would have thought that it would take me effectively over two weeks to finish it?

Along with some peers we came just in time to the Metropol theater where the screening of the stop-motion adaption of Neil Gaiman’s book Coraline was rolling. In stereo. The film remained quite close to the book and featured some very inspired and inspiring designs (keep your eyes open in the garden scene — lovely!). In my opinion the film still got a little too American but hey, it’s a big production after all. The animators did a tremendous job: The cat really moved like a cat and Coraline was most convincingly animated in the top-shot when she creeps into her parents’ empty bed. Further I’m thinking about buying the soundtrack. So if you consider yourself only a minor Gaiman fan and are not following him on Twitter ¹: Go for it, it doesn’t hurt your brains.

Back at the convention center Chris Williams of Disney was talking about story telling in his short Glago’s Guest that I already knew from yesterday. At least that’s what the schedule said. After seeing the short again Chris showed us the final storyboards first, then what story ideas were thrown away along the way of improving it until it worked. He went on to the designs, the color script and the overall style of the film until he showed it one more time. One thing that still bugged me personally is the action of taking out the garbage because it is such a deep rooted American suburbian tradition that it felt really off in the setting of Siberia in 1924. And the garbage can itself was as American as Uncle Sam on 4th of July reciting the Bill of Rights. I will finish my nitpicking on this one by stating that this lecture didn’t really deal with story telling that much.

Richard Edlund ,
originally uploaded by Phil Strahl.

After the lunch break where I enjoyed a chili hot dog, Eric Roth, chairman of the VFX society talked to VFX legend Richard Edlund about his work in the early days of visual effects in movies such as Star Wars, Raiders of the Lost Ark and Poltergeist and also Die Hard and Ghost. Nowadays when everything is so easy and every 16-year old with a computer can make stunning VFX, one forgets that in those old days visual effects were as complicated as they were time consuming. The imploding house in Poltergeist took an artist eight months to rotoscope. Hell! To my regrets this interesting panel passed way too fast.

Syd Mead,
originally uploaded by Phil Strahl.

Syd Mead, the one and only, held his presentation (the same he gave on the Siggraph Asia before) about his rise from early childhood scribbles to the latest designs. His childhood images already showcased his early fascination with cars. It was incredible to see his futuristic visions of the late 1960′s and early 1970′s with car designs that look familiar with today’s eyes. Syd really knows what he’s doing and has a story for every of his paintings. In a near photorealistic rendering of his Hypervan he points to a chrome-like disk somewhere on the outskirts of the painting “This is the security droid in this marina”. Every painting he showed us had a story and he could talk in detail about every detail. Except for the bathroom design for Blade Runner. “Do you know what this is? I don’t either. It just looks like it belongs in this bathroom.” He has funny explanations for anything, not only in his paintings. “Do you know what Gouache means? It’s French for ‘bitchy medium’.”

Habib Zargapour,
originally uploaded by Phil Strahl.

Habib Zargapour was in the unfortunate position to speak after Syd Mead still he pulled it off quite well. From his experience, coming from films to games, he outlined the similarities and differences between designing for games and designing for movies. Still, a lot of principles are alike, yet the biggest uncertainty factor is that you can’t control the camera, so you have to control the environment and make sure it works from every angle. Further you can’t work on a shot-by-shot basis.

“Visual Storytelling in Computer Graphics” by Harrison Ellenshaw had the charm of one of Fraser McLean’s seminars: He talked passionately about the principles, underlining them by showing clips of great movies including ancient Disney features. The films that he showed to the audience (Ryan’s Daughter, Cinderella, Dr. Strangelove or How I Learned to Stop Worrying and Love the Bomb) he commented with such a passion and enthusiasm that you couldn’t help but feeling it yourself: Wow, movies are the greatest and purest thing mankind has ever produced.

Again, this year Shelly Page from Dreamworks brought the fat of the land (mostly France though) of animation to us in the last hour in her “Shelly’s Eye Candy” presentation. Here’s a complete list of all the presented films:

• Yankee Gal, the moments in the life of a WW II pilot in a crashing airplane.
• French Roast, very funny, very French animation about an unappealing protagonist. I love those kind! And the coughing clochard!
• Steel Life, so very abstract and visually strong, like a remake or homage of Koyaanisqatsi would look like. And, no surprise, the music really drives it home. I mean *really*! Composer was Mathieu Alvado.
• Carver Audi Commercial by Framestore CFC. As usual visually very strong and makes you wonder before the payoff.
• Avatar Coca Cola Commercial, also Framestore CFC. The connection to the product itself was totally random, I guess somebody just loved the idea of populating the world with avatars.
• Stork Monster commercial, another Framestore CFC thingie. Very good idea, very well executed. As always.
• Flap Flap, German short about two ravens. If you’re into toilet humour you’ll laugh. In my opinion: crappy (pun intended). Found no link, sorry!
• They Will Come To Town, as seen yesterday. As impressive as yesterday.
• Guerre Naïve, very French with nanoloop musics and, yes, F-Zero countdown sounds about a racing boy. Strange. The French try to imitate the Japanese and vice versa ² in animation. I guess they have a crush on each other — cute!
• For Sock’s Sake, a Calarts graduation animation about a lost sock and his family of other clothes trying to find him. A very fresh idea and witty, expressive animation.
• Ex-E.T., about an alien child that’s just not in sync with his environment. Very good payoff. You’re gonna like this one (or at least the end).
• Dix, about a neurotic’s torment to overcome his compulsion. Very gory and disturbing at many points. Top notch!

Again, I had troubles keeping my eyes open and my mind sharp but I succeeded. Still I had to spend the rest of the evening in my car, sleeping, before paying an fmx party (‘Echtzeitparty’) event a visit with some of my peers. I really don’t like going out. Today I got reminded of that fact yet again.

What I have learned today:

• That it is impossible to find a parking garage that’s more expensive than the one I use.
• That creative argument is the best you can ask for. Any idea only gets better when creative people keep chewing on it.
• That it often helps to get new ideas by drawing without constantly thinking about what you’re drawing.
• That cuts that don’t cut into action are very in your face. If you want it that way, then have the audio have the same harsh cuts.
• That story is about change.
• That you shouldn’t overdraw your storyboards. Only draw what is necessary to the understanding. Then break that down into the least amount of images possible.
• That chroma keying on a chemical basis is like sumo wrestling: You have this huge opponent and you just want him out of the ring.
• That production wise VFX are a tightrope between the producer and the director.
• That you should trust your instincts once you get better.
• That when you draw people in long robes you don’t have to worry about drawing their feet.
• That you got to have a story behind/in your painting, no matter how unimportant it might seem.
• That you get interesting designs when using cliché for you audience to instantly recognize where you are going to take them, then add a new unusual wave to it.
• That constraints help good design.
• That ‘weenies’ in environment design basically are landmarks: They help you navigate the environment.
• That in first person shooters you tell a story basically by how you lay it out.
• That (in games) “story means action” (Habib Zargarpour).
• That ‘Divided Highways‘ is a good book on architecture and, indirectly on level design. So are ‘Learning from Las Vegas‘ and ‘Carchitecture‘.
• That concept art is what helps people to agree on something — thus saves money and time in the end.
• That television is a sales medium. You want people to get involved so they’ll watch the commercials.
• That you should tell as much as possible visually.
• That “action is character”, it is defining the character(s) — (Harrison Ellenshaw)
• That when you move the camera, everything moves. Does your story really wants you to move everything? If not: Keep the cam rigid.
• That “The Curious Case of Benjamin Button” really is scary stuff for actors, but so was “Cinderella”. But…
• …CGI is just another tool that won’t replace real actors or real humans operating those tools.

What surprised me today:

• That George Lucas is said to be rather introverted. Just like his chin suggests.
• That short films by the big studios don’t make any money. In fact, they only cost the production a lot.
• That all the helicopters in Die Hard were added in post. All of them!
• That the movie “Ryan’s Daughter” has absolutely great pictures.

Hotel Hottmann,
originally uploaded by Phil Strahl.

When roaming the Stuttgart streets again it didn’t feel as nice as last year. It was cold, it was foggy and some drunk junk was shouting profanities and bugging people on the Schloßplatz. A sharp turn took me to Starbucks where a friendly caramel macchiato was waiting for me and where I decided today’s program amidst men in fancy suits and a flock of girls skipping school.

I was among the first few visitors who showed up at 9:30 to view the introductory selection of short films from the Filmakademie Ludwigsburg featuring Urs, Something’s Coming, Lebensader, a short for the Cartoon Forum and finally the long version of the fmx’s visual jingle, Frequency Morphogenesis. Conference chair Thomas Haegele bade us welcome and without a transition the first presentation started: “Previsualizing 9/11″ about the previz process of Oliver Stone’s World Trade Center.

John Scheele and Ron Frankel talked about the long and thorough previz on that film because there was no room for stylization because we all know the disturbing pictures by heart; “Documentary footage becomes the iconic reference of an event”. The previz they created was used by all departments throughout the production phase and was like puzzling together what was happening on a grand scale and what the real survivors experienced. “It was understanding what really happened vs. what the two survivors thought they saw”.

It was not possible to shoot on the real Ground Zero for all the terrible memories the scenes would evoke, so the production needed to pursue a different approach. High resolution HDRI photographs were taken from the surroundings so they could be used to populate the digital recreation of the site. The film was entirely shot in Los Angels, partly on Lebanon Street, the only street that looked somewha Broadway-ish.

The previz was divided into a practical previz for the different departments e.g. what the camera crew needed to know, the set decorators and so on, and into a post-viz meaning where buildings needed to placed correctly after the shoot was done. Ron Frankel re-created a large part of Lower manhattan in XSI up to the details needed for getting the big picture as well as what the survivors saw — they believed until their rescue that a bomb went off in the garage.

My ticket,
originally uploaded by Phil Strahl.

I skipped “A Global Production Pipeline” Xavier Nicolas from Lucasfilm Animation for some more java at Starbucks before returning just in time to Sony Imageworks’ “Animation and VFX” by Bob Osher from Sony Imageworks. His presentation felt at first like its target audience was potential shareholders and the emphasis on “Innovation in Support of the Filmmakers” sounded as cliché as does the slogan “The Future — Now!” ¹. Thing got a little more interesting when the Arnold renderer was briefly touched, although a little too sketchy but what really blew me away was when Bob introduced Sony Imageworks’ inhouse post-production tool Katana ² which I understood as an optimization tool that interconnects 3d and compositing back and forth and saves big amounts of time and, effectively, money. In the course of the presentation I saw a the node tree of the wide shot in Watchmen where Dr. Manhattan blows up, well, Manhattan that was also done with Katana. Speaking as a Nuke compositor I have to admit that it made me kinda frisky.

Generally speaking: Sony has a lot of sophisticated in-house tools to help the artists and is very proud of their upcoming feature Cloudy with a Chance of Meatballs. Bob showed us the Jell-O scene and after 30 seconds he won us all for it. Can’t wait to see it in “mouthwatering 3D”.

Now we all were hungry so I got me a nice hot tomato soup with bread for lunch and roamed the stretching shopping boulevard, eager to find some place where I could by a shaver and some eyeliner.

After the break followed a little panel titled “I Got A Job Abroad… Now What?”
hosted by Jan Sjovall and featuring thee more Germans who made it abroad. The room was already full when I arrived so I was cramped into the back and sat rather uncomfortably close to the floor and the informational value of the panel was scarce. Still they dropped a few things to consider when working abroad like that you only realize in comparison how different your own cultural background is.

Over the day I met some folks of rise fx where I spent last summer four months as part of my internship. It was fun chatting a little and so I decided to see their presentation that was part of a broader presentation of the VFX and animation facilities in Berlin-Brandenburg. I already struggled a little with my sleepiness — four hours definitely are too little.

Watch ‘em

Haus der Wirtschaft,
originally uploaded by Phil Strahl.

I went up to witness the last minutes of Alex McDowell’s lecture on the Production Design for Watchmen that was followed by John ‘DJ’ Desjardin’s presentation “The VFX of Watchmen”. Those guys are crazy!
I kept fighting against dozing away and luckily I won mostly because “Making of Dr. Manhattan” by Pete Travers from Sony Imageworks was very intriguing: From start to finish it took about nine months to develop the character visually as well as technically while keeping the VFX footprint on set as low as possible.

“If you have developed the best tracking system in the world but it hinders the actors you end up making perfect tracks of bad performances. Which are totally useless.” Pete said.

The actor playing Dr. Manhattan was wearing a suit covered in blue LEDs and tracking patterns and so he became not only source for video motion capturing but was also a very prominent practical light source on set. The tracking was done by triangulating images from the film camera and two Sony HD video cameras that captured the action from two more angles. The biggest problem in this approach was to sync the shutter phases of both the video cameras and the film camera to make tracking possible.

Another problem was the hue falloff of the LEDs: Close they where cyan, further away their light color became indigo. Hence the actor in the suit needed to be painted out on a frame per frame basis it became a huge amount of paint-work because he was a light source and every frame had to be painted separately instead of having a clean plate. So talk about painting hell.

While most of the people went directly to the Metropol theater to see Watchmen I really needed a break from all the highlevel-VFX and stayed for the “Animation Show of Shows”, introduced by Ron Diamond of AWN. I can get Watchmen on bluray at any store but might won’t be able to see some of this great animated films again. So here’s the list. I hope I didn’t mess up with the French titles.

• Keith Reynolds Can’t Make It Tonight, a witty stickman Flash animation that shares a lot with xkcd both visually and narratively.
• La maison en petits cubes, a hand drawn animation that tells the story of an old man rediscovering his past. A Japanese animation that looks totally French.
• KUDAN, a very abstract CGI animation about the relationship of a father to his child. Japanese. Weird. Breathtaking.
• La Queue de la Souris, a short minimalist tale of a mouse trapped by a lion. French. Witty.
• I slept with cookie monster, an analog animation drawn with pastels that tells the story of the animator that was abused by her lover and how she dealt with it.
• Franz Kafka: Ein Landarzt, probably one of the weirdest animations I’ve seen lately. Truly, the Kafka-esque spirit was captured very well in this short film.
• Glago’s Guest, Disney’s computer animated short of Russian guard Glago watching over endless Siberian snowfields.
• Hot Seat, The Office meets children’s cartoons. Funny yet true.
• Presto, a Pixar short I won’t get into because all of you know it already.
• Skhizein, my favorite today. The story of a man who is always 91cm besides himself.
• KJFG No.5, a very short animation where you’ll go “wtf?” at first. It is about a band jamming together that gets disturbed by a hunter. Great ending!

That was my day.

What I have learned today

• That foamcore models still are a big part of previz as are low-res interactive environments that every department can access.
• That it probably was a bad idea to jokingly refer to my new styling as “racoony” once — the word spread.
• That on very documentary films it is necessary to make sun and moon studies.
• That Sony’s Katana is da shit!
• That at Sony they deliberately decided against a house style and that they “challenge every assumption”.
• That Germans only realize how German they are if they work abroad.
• That you should start with 3d as early as possible in your previz for any shots that are not static.
• That a good way to ensure consistency in applying tracking-dots on an actors face is to make a plastic mask from his face, drill holes accordingly into it and then have him put on the mask: Make the dots through the holes and you’re done!
• That your VFX tricks on set should do anything but hinder the performance.
• That the scanning of skin textures should be done when the skin is anything but perfect or else you get the typical too-perfect-to-be-true CG-look.
• That eye moisture helps a great deal in the believability of a CG character.
• That instead of simulating rimlights in your shader (I consider that a no-no anyway!) you need to take the extra mile of adding peach-fuzz to your digital character. It renders longer, but looks much more convincing.
• That I really have to get my sleep cycle straight before attending the fmx.

What surprised me today

• That Oliver Stone looks like a chubby Albert Speer. Creepy!
• That my geekiness in terms of comic books is way below what’s common in the industry.
• That Zach Snyder draws really good.
• That I can sleep rather well on the floor of my car.

The Russians at my hotel made a lot of noise, chatting on the hallways until 4am, chatting in the hallways from 7am on and using the floor’s toilet ¹ every half an hour. But I was so exhausted from the evening at the Schräglage that I slept well for the first time.

The Plan

Turkish Coffee

That’s what I wanted to do,
originally uploaded by Phil Strahl.

After checking out I was way too early so I visited the Starbucks once again where I wanted to check my email (which piled op constantly — unfortunately not with responses to my applications). I ordered a tall Iced Caramel Macchiato and went to the counter where also some teen with Turkish background was talking to the Tunesian barista giving out the coffee. Somehow I got involved in the chat and the barista said

I nodded

and so I spent twenty minutes chatting with the two guys, the barista derogatory talking about women and his girlfriend using terms I won’t reproduce on behalf of decency. The Turkish teen, after asking about my strange accent (= Austrian), asked me a question on his mind in such a straight forward way that I nearly chocked on my macchiato.

Pause.

That was a crapload full of very sensitive questions so I took my time to explain it all very detailed and as neutral as possible. ²

the barista said, threatening with his fists; people turned around. It was time to gently switch the topic to something else.

I asked the teen but the barista replied

the teen didn’t pay attention to the raging barista.

he pointed absentmindedly onto the floor. Grey tiles. One was cracked.

That was hard! I asked him whether he had sent out any applications and he nodded.

the barista looked up from behind the counter, opening his mouth for some reply but got distracted by a blond girl in a mini skirt passing by outside. The teen asked me about my admittance card that was hanging around my neck. I told him a little about the fmx and the companies that came to my mind he might know: Pixar, DreamWorks, Disney.

It rang a bell and for a secon he drifted away in memories, probably of a film. I seized the moment:

He shook my hand, the barista was grumbling something and I thanked him for the coffee. Then it was time for me to get to the first presentation.

Bread and Games. But mostly games

fmx flags,
originally uploaded by Phil Strahl.

Videogames Today and Tommorow’s Gamers by Joseph Olin, president of the Academy of Interactive, Arts & Sciences, who was the host of the day’s other presentations too. He just showed a couple of statistics and demographics followed by a short history of innovative games:

Zork (1980) as an example for immersion, Tempest (1980) for adjustable difficulty, Contra (1988) for the first cheat codes, Prince of Persia (1989) for it’s realtime challenge and score, Lost Vikings (1992) for multiple characters in the gameplay (by the guys who later became Blizzard) and Flashback (1992) for its handdrawn backgrounds and animation.

He also mentioned Into the Pixel, an opportunity for game artists to submit their works for exhibitions about the art of game making.

Animation Systems on Assasin’s Creed by Sylvain Bernard from Ubisoft really got into the nuts and bolts of the animation system they used for creating a blend of motion-captured animation and keyframed animation they call “stylized reality” animation. They tried it totally realistic at first but the ore real the movements were, the less resposive was the control over the movement, e.g. when you stop pressing forward on the gamepad Altair would have had to finish the animation cycle he was in first, then going into a “stop” animation which would total to about one and a half seconds of movement not controlled by the player.
And there were a lot of animations for the three (or five?) animation artists: Alone for the ground movement existed 168 different animations.

Collision detection and prevention during fights as well as making sure the involved characters were in sync to the animation was achieved by them sharing the same reference “pivot” during the animation.
Cloth movement was done procedurally for Altair, for all the NPCs there were hinge bones without collisions used as handles for the forces.

Altair shared his skeleton with every character in the game. This made it possible to use the animation in a modular fashion: If Altair’s ground-movement fit also for a blacksmith you could keep it (saving memory), the same time you could say the blacksmith NPC has a different set of fighting animation.

Although I had troubles staying awake I realized how much work and how much brain power was in the final great looking game. Can’t wait for the PC version!

Developing Games with Pixar was canceled thanks to some tragedy in the speaker’s family. So I went to (a different) Starbucks, guzzled some Strawberry & Cream Frappucino before returning to the convention for my very last presentation of the day:

Visual Design in Video Games by Ryan Wilkerson from the Microsoft Game Studios. I was expecting some artistic guidance but he mostly spoke about the perfect pipeline, from finding references to the final shipped product, along with some description how the engines render stuff, namely in passes — much like we do for our 3d-projects.

Ryan had to cut much of his presentation shortly before because he got a call that most of the conceptual stuff he wasn’t allowed to show wasn’t announced yet and hence top secret. But from what he mention in his answer to one question about emotional involvement of NPCs and that “Molyneux will kill me if I tell too much” I am making a wild guess that the top secret un-announced project will be Fable 3. A very wild guess though.

That was the FMX for me, Donsch and Joschi who were my passengers for the drive back to Salzburg. I really was expecting heavy microsleep like the one I had during the ILM presentation, Carlos Baena’s animation show or that day’s Assasin’s Creed display of excellence. But everything went fine, apart from the very dense traffic slowing us below the recommended speed for the whole journey, so it took us exactly 4:30 to get back. Without any microsleep!

Speaking of which: I gotta get some macro sleep…

What I have learned today

My Yaris parking for 20 €.,
originally uploaded by Phil Strahl.

• How Austria is perceived by the world outside.
• That three different coffee beverages are too much before the first presentation.
• That old is new (in terms of game ideas).
• That a simple basic gameplay crafted very well is key.
• That casual games have much in common with short films.
• That directing a game is the Director’s Vision vs. Player Choice: They exclude each other.
• That setting up everything (everything!) modular is a good way to avoid redoing a lot of work.
• Modeling detail textures to use as normal maps is probably faster than getting them right by drawing them in Photoshop as bump-maps.
• That content creation for a game is much like cooking where you have a recipe (concept), ingredients (assets), your kitchen (development tools) and your finished dish (runtime). Now you hope everybody’s happy but there’s no accounting for taste.
• That Microsoft Games is producing 95% for the console market and only 5% for the PC market. And they aren’t the only ones. Tough times ahead…
• That the DOOM-generation (in terms of game-makers) all got kids themselves now and don’t want to make games that they wouldn’t allow their (pre-)teens to play.
• That procedural animation will be a big topic in the following couple of years.

Because I had lost my Ratatouille poster last night I decided that the best thing to do was to attend another RenderMan presentation, get another walking teapot, get another RenderMan sample DVD and to fill out another survey sheet. In the comments section I apologized for showing up again only for a new poster. Sometimes you just got to be honest.

The Plan

The people from the jumping lamp studio

Morning in Stuttgart,
originally uploaded by Phil Strahl.

But instead of hearing the same stuff again about the integration of Massive into the Pixar pipeline by Ziah Fogel there was Afonso Salcedo, one of the lighters at WALL·E who did about 3 or 4 percent of the shots in the whole movie, “a good average”. I learned much about the way of lighting at Pixar, and how they dodge using ray tracing, the Achilles heel of RenderMan. For WALL·E the director wanted a 70s sci-fi movie look so Pixar got Dennis Muren over to set up some of the basic lighting for WALL·E in the atrium with a 70mm-film camera with anamorphic lenses and some WALL·E and Eve mockups in real size. They did tests on the DOF, the cushioning and the way lens flares and iris circles look. That’s what I love about Pixar: Total involvement and research for every project.

I asked Dylan Sisson (?) afterwards and he answered me quite sharp that they don’t because they want to keep the full artistic control over the light ¹. They even light characters and background separately. So they set everything by hand, about two hundred lights each shot. No wonder it takes them so long!

When the presentation was over I got myself a nice espresso at the lounge and returned half an hour later to the recruiting presentation which was in fact a Q&A session with all the folks from Pixar that were present in Stuttgart: Carlos Baena, Sharon Calahan, Dylan Sisson, Afonso Salcedo, David Munier, and Ziah Fogel. I seized the day and asked them a lot about anything that came to my mind, most importantly how they do their compositings and what software they use and whether they are looking for a compositor.

That hit me deeply. They don’t even have the job I was hoping to apply for.

After the session there was a buffet in the little room outside the seminar room and the audience mingled with the Pixarians. Paul from the 4th semester was interested to learn from Carlos how the animation of WALL·E’s crawlers played a role in conveying his character. I was still eager to get some of my questions answered by Sharon Calhan, Carlos Baena and Afonso Salcedo at one table – unbelievable. And silly me forgot to take a picture before I left for some presentations with “educational” value for me.

3D and Live Action in 3D. In stereo

Meeting Colleagues,
originally uploaded by Phil Strahl.

Stereoscopy is the buzz of the year, as I mentioned earlier. Disney, Pixar (yes, there’s a difference!), DreamWorks, ILM and Sony ImageWorks are working on features in stereoscopic 3D now and all of us artists in the industry either adapt or become extinct. I chose to adapt but was very skeptical about it.

Journey 3D by Bret St. Clair (from Meteor Studios) and Chris Harvey (from Frantic Films) was the first presentation in stereo ³. The two studios in Canada have been working on The Journey to the Center of the Earth 3D starring Brendan Fraser and had to handle a live action/CGI feature to seamlessly blend together – all in stereo.

The two of them talked about their struggles and problems and seemed as if they were telling a terrible tale of how they had survived 9/11 and Katrina. They had to tell so much in such a short time that I noted down like crazy what I felt was important. And there was a lot, starting from the camera rig on set. They were using two different rigs, one that allowed to adjust the interocular separation ⁴ even to zero if they had wanted it to thanks to a 45° semitransparent mirror (which produced differences in distortion and lights), the other rig had an interocular separation of 2.7″ as opposed to the 2.5″ space between the average viewers’ eyes.

Zooming, for example, was nearly not possible because the two cameras had to be in totally in sync which was not possible in live action shooting because even an offset of the tenth of a pixel makes a difference to the perceived depth.

Matte paintings should be executed at least in 2.5D, but it is much better to project them onto geometry which shouldn’t be too low-poly because you just can’t fake it in stereo. And don’t paint haze, fog or atmosphere in your matte paintings because that all needs to be done in 3D.

You can’t make a 3D film into a stereo 3D film at one point, you have to be certain about the presentation when you’re working on the concept because from that point on, nearly everything should be executed in stereo, especially the layouting and animatic. Artists that have only worked on non-stereo pictures before (which is the majority of us at the time) tend to cheat to get the image an aesthetic quality instead of being 100% correct in a three dimensional space.

Create optimal viewing conditions. From the emphasis put onto that statement it is the most important one. You have the same projectors on set and in your studio as you will have in the theater, you have to have at least a 20 ft screen for the projection and there should be one 2k stereo-projector instead of having two separate projectors because you will have to stop the footage, scrub through it, etc. to review your work.

Stereo means to double everything: Storage space, amount of footage, time for trouble shooting, compositions and rendering time, when you’re not working with some tricks: In the case of Journey they were able to have the second image rendered in only 20% of the time of the first one by caching a lot of render-data so only the occluded geometry had to be re-rendered. Pretty clever and it saved them a lot of precious rendering time. They used Gelato by nVidia for that, but I have no idea what it exactly does.

And now for the dreaded, feared and gruesome focus on matchmoving in stereo *insert 50′s horror film music here*. Even for ground planes it had to be done absolutely correct to every 0.1 pixel otherwise something would’ve been off in the final depth perception. A tracking cube helps you much in those cases because the perspective its clearly visible at all times. After tracking you should test it in the theater-situation before continuing because it has to be absolutely correct. The guys developed a way of tracking only one view of the stereo and then calculating the correct track for the other “eye” because tracking them separately didn’t result in absolutely the same movement for each eye.

Keeping as much meta data from the camera as possible in each shot. You might not use it at all times but sometimes it really saves your shot.

Another thing to know is that distortions work differently. Instead of having a centered distortion for every eye, the distortion should be in the middle between the eyes, making them off center for the individual camera footage. Lens artifacts should be matching in too: It is distracting, if you have a lens flare in only one of the views or that highlights don’t match and hence “flicker” when looked at. If you can’t avoid this on set you have to either remove the highlight in question or digitally double it for the other eye.

When it comes to compositing you want to have both eyes’ footage in a single composition, although it doubles the complexity. But you can make corrections to one eye where needed and have them share nodes and effects at the same time. Ideally the compositor should be sitting in front of the 20 ft so s/he can always check what it’ll look like in the theater. “Awake” for Fusion does the job ob having both eyes in one composition, by the way.

When evaluating stereo depth you should always revise shots in sequence so you can avoid too harsh differences in depth that the audience’s eyes don’t need too long to adapt each shot and might miss the important action.

Oh, and bump maps should be traded in for displacement-maps!

3D in 3D

Stereoscopic Depth as a Storytelling Tool by Mark Empey and Robert Neuman from Disney Animation was a very insightful presentation because there literally were worlds between their stereo-footage and the previous presentation. Disney showed off a five minute long montage of the stereo version of Meet the Robinsons and the depth perception worked just fabulous, seemed harmonic and believable. The first seconds were terrible, the following 20 seconds were okay and after the first minute I never wanted to watch plain mono footage ever again. Journey looked 3D-ish but not quite that pleasing. Even I could tell, that Disney had mastered 3D as they had mastered 2D. And they gave an insight about their way:

First of all you should avoid using stereo like it was used in the fifties and eighties:

Making it comfortable for the audience is the most important thing. If they get headache, hurting eyes and nausea from watching your stereo movie they will never want to watch one again. So you should keep the interocular distance of 2.5″ in mind. That’s not so much important for near objects that are “floating above” the screen, but for those who are beyond infinity: When looking at objects that are on the horizon both eyes are exactly parallel. But with the use of floating windows (more about them below) it might happens to have the audience look beyond that point where the eyes aren’t looking just parallel, they’re looking outwards which is painful because mother nature never wanted us to look that way. So avoid having offsets bigger than 15 pixels towards infinity.

Just like using a color script it helps much using a depth script where a value from one to ten (sometimes eleven) tells the stereoscopic supervisors how deep a sequence, scene, shot has to be. Neuman (or was it Empey?) said that for him stereoscopic depth equals with emotional attachment to the character. So a depth script is essentially a script where the arch of emotional depth is traced. The closer you want your character to connect with the audience, the more you push her/him out of the screen (not out of the floating window, to be described soon!). For emotional distance keep your character “behind” the screen.

Sometimes you want to have objects come out of the screen very close. It’s okay if you do it in the middle of the screen but when one of the stereo images is being clipped by the screen frame, then a “window violation” occurs. When this happens a viewer perceives the presented images much flatter because her/his brain needs to constantly decide between two conflicting different depth-cues. So what do you do? You produce a “floating window”.

The floating window is created by unequal masking of the different stereo-images, e.g. you give the left image a thick black border on the left and the right image a thick black border on the right. This little trick lets the screen in the theater appear as if it was floating in front of the action, depending on the thickness of the black border because the viewer always associates the floating screen with the physical one.
You can even have the floating window rotate and tilt, all with diagonal masking. Disney uses a Maya plug-in that displays for the stereo camera rig in the viewport the near plane (where nothing can be any closer without risking eye strain), the floating window transformations, the screen plane (where the physical screen in the theater is located) and the far plane (which defines infinity) and can be controlled by eight values.

DOF should be limited and only applied to objects in the background (“behind” the screen). For objects in the foreground you should deemphasize their importance with lighting.

Don’t toe in the two cameras in your rig, because this results in vertical mismatching and is uncomfortable to view. Instead keep the cameras parallel at all times and use only a different portion of their perspective correction lens works.

At last it is important to have your characters round enough to avoid that your characters look like cardboards which occurs when the roundness of a character drops below 20%. This occurs when using telephoto-lenses where a lot of depth-information is eaten up by blank space. One way would be using a wide angle, but a better way it using multiple camera rigs, one for the foreground with all the necessary depth information and one for the background. So you can skip the blank space between character and background and use this range to enhance the roundness instead.

Disney used all these techniques for their upcoming feature BOLT and from the very raw work-in-progress material that was screened I can say that it works marvelous.

French Stench

Sorry about that headline, I just like cheap rhymes (but no hip-hop!). The presentation titled Effects Tailored to a Director’s Vision by Pierre Buffin and Francois Aubaque from BUF Compagnie sounded rather interesting, because BUF use only proprietary software since they started in the eighties and have grown in technology as well as in style and expertise. They did the effects on Fight Club, Panic Room, Silent Hill, The Prestige and any Astérix and a heap of others. So I was quite thrilled to learn about their work with the directors such as David Fincher.

As it turned out they were talking English with such a terrible French accent that it was hard to make out what they were talking about. But instead of telling much they just rolled making-of after making-of the different productions and just said a few words between them. I was quite disappointed because I can watch all that making-of-stuff alone on any special-edition DVD of the given movie (even without the accompanying French songs). So I left after twenty minutes for another caffeine flash to get ready for Carlos Baena‘s much anticipated Realistic vs. Stylized Human Animation presentation.

He showed his work on Star Wars Episode II as example for realistic animation and some shots of Skinner in Ratatouille. The shots he worked on are also on his website. He also showed a short animated clip from the Iron Man Trailer, the shot with the iron man’s reaction as the tank misses him: He dodges, leans back more than he had anticipated so he has to take a step back to keep the balance. He lifts his arms, looks back where the shot came from and shoots back from the gun on his arm. The rebound travels up his arm and reaches his head an chest after a couple of frames, so he shakes a bit, the left arm extended into the other direction to keep balance. Now was that a thorough description of a little animation or what?

Then he showed some reference footage he uses as an animator, a DVD with slowmotion footage from the Olympics in Japan some time ago where you can see most of the actions from the side or the top. Some other references are x-ray films, e.g. of a man raising his arm.

He looks into the audience and continues honestly.

He showed some more footage in slow-mo, of a fist hitting on a table, where you could see the muscles and the skin ripple on the impact. That’s the stuff you should be watching all the time when becoming a character animator!

In terms of stylized character animation Carlos dug up some early tests he did on Skinner, the cook from Ratatouille who was based largely on Louis de Funés, my favorite French comedian. Just watch his performance in Jo and you’ll see why the folks at Pixar decided that Skinner became too French.

Unfortunately the amount of coffee I had pumped into my veins that day made me rather immune to any more caffeine so I had troubles staying awake during the whole presentation. When Carlos was talking something about Woody’s Arms in Toy Story I drifted away for a couple of seconds, or a couple of minutes — I don’t know.

The presentation titled Bringing Nuke and Maya Together by Andreas Frickinger and Daniel Hasenbring from Miromar was held in a way too small room for way too many interested people so I was among those who sat on the floor, when the two Germans showed their tricks of rendering a Mental Ray mia_matrial in a crapload of passes and having a script rebuild the shader in Nuke again, where you can tweak on every nut if you ever wanted to.

The most stunning part, however, was the possibility to re-light your rendered scene within Nuke because their pipeline also exports UV and World coordinates. The only drawback is that your Nuke-lights will not cast any shadows but for subtle re-lighting it’s perfect.

Even some of the scene geometry can be viewed in Nuke’s 3D view — it’s crazy how such an amazing workflow-demonstration came off like a high-school presentation of two totally not entertaining people that seemed not too well prepared either.

Afterwards I talked with Kerschy a little who also witnessed the presentation. He was so amazed about the whole topic that e decided to write his diploma thesis about how to get from Maya to Nuke like the guys from Miromar.

What remains as item on a list of impressions:

Morning Impressions,
originally uploaded by Phil Strahl.

• My first real meal this week, enjoyed at one of the local McSteakhouses, Maredo.
• Strolling around the heart of Stuttgart without knowing a place to go.
• My visit to the Schräglage-lounge with lots of goofiness, caffeine and laughter.
• Another day without warm water at the hotel.

What I have learned today

• That you can’t cheat when working in stereo.
• That zooming is not possible in live-action stereo
• That subpixel-accuracy matters to the tenth of a pixel in stereo
• That matte paintings should be at least in 2.5 D when working in stereo
• That you can’t paint haze into the matte paintings when working in stereo (= faking it)
• That even the layouting of a stereoscopic shot/scene/film should be done in stereo
• That you should be able to view your dailies the same way your audience will watch the final stereo picture.
• That you should create the same conditions on location for viewing stereo dailies as your final audience will have.
• That camera shakes can be used (moderately!) to hide stereoscopic issues.
• That stereoscopic 3D-rain looks fake in Journey. Really fake, just big blobs dropping.
• That when working in stereo you need to double nearly everything in your production calculation.
• That matchmoving is nearly impossible for live action stereo footage.
• That using a cube for tracking stereo footage is much better than using single tracking marks without any depth information.
• That you should collect as much meta data from the cameras as possible when working in stereo.
• That distortions in 3D share the same center, which lies between the “eyes”.
• That lens artifacts should apply to both eyes. If they don’t match make them match.
• That a composition for stereo shot should be done for both eyes in one composition.
• Stereoscopic depth should be kept persistent throughout a sequence.
• That you should use displacement maps instead of bump-maps for stereo pictures.
• That you just can not cheat when working in stereo!
• That you shouldn’t use stereo for gimmicks
• That you have to make watching stereo as comfortable as possible for the audience.
• That you should avoid “beyond infinity” focusing on your audience in stereo. 15 pixels at 2k are enough!
• That you should use a depth-script when working in stereo.
• That perceived depth can equal emotional depth.
• What “floating windows” are in stereo and how you can work with them.
• In stereo DOF should only be used for the background, for the foreground deemphasize with lighting.
• That you should use a different portion of the image circles of parallel cameras in stereo instead of toeing them in.
• That you can counter cardboarding in stereo by using two camera rigs, one for the foreground, one for the background.

The Plan

My rustic hotel room,
originally uploaded by Phil Strahl.

The day started by waking up from some weird dreams in my hotel room. I had breakfast downstairs where a very talkative woman mistook me for some Formula 1 pilot I don’t know, probably because I was wearing my leather jacket. I enjoyed a yogurt and some coffee along with buttered bread.

After looking for a cheaper place to park my car I ended up again in the most expensive parking garage that was so comfortably close to the FMX building. But I was too soon, so I stopped by at Starbucks, got myself my beloved caramel macchiato and spent the rest of my remaining time looking at the day’s program.

A Double Negative on Internships

I felt a little alone that morning (not that it bothered me much but I just needed somebody to chat with) but nobody showed up. So I sat myself into the small room with the recruiting presentations of Pixar, Trixter and Crytek – the room was cramped with upcoming character animators and people crazy for Crysis so there was much expectation in the air. The nice lady of Pixar showed some photos of the site located ten miles west from the Golden Gate and that they were looking for technical directors and was done after about ten minutes.

Trixter is a Munich based animation company looking for animators too but their demo reel really looked humble after Pixar. Crytech was looking for “anybody passionate about games” for some upcoming and secret project“ and that was in fact the first hour of recruiting presentations. I talked a little with the lady from Pixar about what kind of TDs they were looking and it turned out that Pixar was in need for anybody interested and talented and that I should drop by at their recruiting table.

Double Negative Recruiting,
originally uploaded by Phil Strahl.

The following hour Double Negative and Pixomondo Studios were holding their presentation. Kerschy, a fellow student, classmate and friend interested in shading, lighting and rendering sat next to me in the first row when two ladies and a gentleman all from the London based VFX facility Double Negative showed off their portfolio and some breakdown of the Cloverfield compositing which was mainly a rotoscoping and matchmoving hell because it was all hand-held camera with actors against an all green background – even the ground they were walking on was green. It just looked impressive in the final. When Pixomondo started their presentation Kerschy and I left into the recuiting room when Adam Donavan called with some news about his game Frontiers. Kerschy was getting more and more impatient because other people dropped by and took the seats in the Double Negative booth. Finally I managed to finish the call quickly and Kerschy and I talked to the very nice and very sexy Hannah Acock about a possible internship at Double Negative.

Now you know why I am always sitting in the front row whenever possible: People feel like they already know you but mostly can’t exactly tell why. This is a good way of making a subconcious first impression.

she said, a polite way of saying that Double Negative wasn’t looking for green interns. However she told us about career opportunities, Kerschy interested in 3D, me in compositing, as always.

She gave us her private email address, kept our demo reel DVDs (yay!) and wished us a nice day. On our way out Kerschy talked me into applying at Pixomondo and the lady sitting there all by herself with her iBook seemed rather fond of us showing interest. Kerschy was a little disappointed because he trained himself to a Maya/MentalRay specialist whereas Pixomondo has a 3D Studio Max and vRay workflow. But as a compositor it didn’t matter to me where my render-passes come from. I showed her my demo reel and we talked it through in realtime.

I said, noticing instantly a subtle change in the lady’s eyebrows.

I pulled my head out of my statement and it was fine. We shook hands, thanked each other and off we were. Nearly.

A student from the University of Applied Sciences in Ludwigsburg approached me with some nice words about my demo reel and where Kerschy and I come from. We had a little chat about our colleges, shared some advice but then it was 12 o’clock: It was time for Curry!

Old Spice

DreamWorks Recruiting,
originally uploaded by Phil Strahl.

Dan Curry was holding his presentation VFX for TV” in the big room and I managed again along with Jonny and Kerschy to get seats in the front row. Dan, who reminded me a little of a short Dustin Hoffman was a VFX supervisor at one of my most favorite childhood TV series: Star Trek: The Next Generation and also did the shots on Deep Space Nine and Enterprise. I haven’t seen a single episode of it and from what Dan was telling, the writing was terrible.

Dan asked the paralyzed audience.

I thought to myself. For an old trekkie like me it was so obvious that I really felt too insulted to shout out “NCC 1701″ so I missed out on the prize. Why wasn’t he asking for the Voyager’s number? I bet not many people would’ve known the answer, NCC 74656.

If I had any fear of public speaking left I would’ve died in that moment because I really wanted to know how the transition from old-school optical printing to 3D evolved for Curry. He answered me that the crystal (I guess in the first Season of TNG) was done in 3D because they couldn’t figure out any other way to do it.

Afterwards I walked up to Curry and told him that I want to shake the hand of the man who got me inspired to work in the field of visual effects and that I was really grateful for that. He looked a little surprised but flattered nevertheless. I would’ve liked to talk to him a little longer but DreamWorks were holding their presentation in another room again.

DreamWorks seemed to consist solely of a couple of middle aged women looking for people to join them in their new facility in India for their upcoming stereoscopic 3D films. Their sneak previews ¹ Kung-Fu Panda, Train Your Dragon and Monsters vs. Aliens which I really enjoyed and recommend to you fun loving readers of my boring blog.

Without any breaks I headed straight into Ethics in VFX by Van Ling and Gene Kozicki but had missed the first minutes. It was a really good and entertaining presentation about what can and what has been done with image manipulation an its effects on society and politics. Makes you really think about your profession where you try to have everything as convincing as possible.

More Curry?

VFX: Problem Solving on The Run was replaced by a presentation of Sony Imageworks (which seems to consist of even older ladies who seemed like a weird lesbian couple to me at first) telling that they move their facilities where the talents are. Just like DreamWorks they have opened up a site in India and were making their first steps in stereoscopy – just like DreamWorks.

Every major company seems to outsource to India because its cheaper. They are looking for Europeans willing to move there too where the salary is above Indian average (so you could pursue a rather wealthy lifestyle there) but probably below American average (which effectively means you have to stay in India until/for your retirement because you couldn’t afford living in the US or the EU again). I was a little pissed and walked out to see any of the DreamWorks ladies.

They were swamped by interested applicants clenching nervously their demo reel DVDs, eager to put them onto the growing pile in the middle of the table. DreamWorks isn’t looking for interns at the time because they feel their effort and time wasted on somebody who’s only there for a couple of months. Still I handed Shelly Page my portfolio DVD, recieved her card and off I was – to Starbucks for another tongue-sizzling caramel macchiato and a mango chicken wrap.

Render, man! Please!

Pixar’s Gift Set,
originally uploaded by Phil Strahl.

I skipped Adobe’s Computational Photography for Pixar’s RenderMan’s technical magic, a presentation a little out of the ordinary. It was held in a small and far off room on the first floor — you only would find it if you’re really interested. Although it was highly technical it never got bored. Perhaps thanks to the little giveaway the Pixar folks placed on every chair: A walking teapot along with a RenderMan demo DVD and a survey sheet you could trade for a Pixar poster when leaving.

I guess that was one reason why nobody left early: greed.

Ziah Fogel talked about how they managed to integrate Massive into their RenderMan pipeline via Marionette: The animators animated a couple of loops and action, e.g. rats squeezing themselves under a door, or just applauding. The data from Marionette would then be transferred to Massive where the facial animation still existed invisibly. After doing the crowd animation they bounced the data back to Marionette, even though it was possible for Massive to produce rib-files ² but they wanted to be able to make changes to the crowd-animation when needed. Of course, there needed cleanup to be done because Massive
made every frame a keyframe.

The second part of the presentation was a hands-on tutorial by Dylan Sisson in Maya how to create an Über-Sprite: It was a plane with the texture of a rocket, that he displaced via the z-Depth information of an already modeled rocket. It rendered quite fast for that, but it was hollow when seen from behind. No problem! Dylan added a “Double Shaded” attribute to the sprite and it was a full 3D rocket out of a sprite that looked exactly like the modeled rocket.

Dylan said and hit “Render” on his notebook, and after 22 seconds RenderMan was done. That was real fast for thousand displacement maps!

Kerschy, who sat next to me in this presentation, and I knew: 3D motion blur takes forever to render, it multiplies the rendering time for a frame by about 10 to 20 times. But how long took it RenderMan for the same view of the thousand rocket-sprites with displacement, only this time with motion blur? Twenty seconds.

Kerschy was eager to know. Not without a little pride Dylan explained to us that RenderMan knows what details that the eye won’t see in motion and omits these in the rendering.

The little surprise after the presentation was a plush toy from the Pixar short “Lifted”, a RenderMan baseball-hat for the person who had a key chain taped under his or her seat. Of course, the key chain was a present too. Pixar obviously makes good money to spend. I traded my survey sheet for a Ratatouille poster at the exit and went with Kerschy for an espresso and a blueberry muffin at the lounge. I have to add at this point that Pixar really grew close to my heart in the two days.

Rising opportunities

Phil Strahl, unemployed,
originally uploaded by Phil Strahl.

Kerschy, as it turned out, had spent most of the day applying and showing off his showreel to various companies and talked me into applying at rise fx.

I took a sip from my espresso that was really good and not that expensive either.

Sounded as if he was really serious about it, so I said that I would try it and finished my beverage. I wanted to drop by at Adobe’s presentation of Computational Photography but neglected the idea of spending the last fifteen minutes of a presentation and so I was perfectly in time for Technologies Invented at ILM Through the Years by Christophe Héry.

I wanted to take my favorite seat again, front row, third seat from the left on the right side. But Kerschy looked at me from the third row, pointed at two people in black shirts, chatting

The guys from rise fx! We switched seats and I rummage around in my bag for another of my demo reels and got one. Then I turned to them.

My shortest application ever, nearly ten seconds. Haven’t heard from them since, though…

Of Calamari Pirates and Car Robots

Max & Yuhsuke passing by,
originally uploaded by Phil Strahl.

I don’t remember much of the presentation because it has been an exhausting day and the clips Christophe showed from the various stages of production were completely silent, so it wasn’t hard to doze off just after learning about the problem of having more polygons than pixels in the final frame in Pirates of the Caribbean 2. I guess that’s what happen when you let your artists go crazy in ZBrush.

ILM likes to work with dirty footage, which sounded a little stupid at first: What kind of compositor likes having smoke, rain and fog already in the raw footage? But on the second thought it’s much better than e.g. adding some cheesy particle effects in the post, because real fog looks real and behaves real. And convincingly adding a digital character into a real smoke tail is not as hard as creating the perfect smoke tail in the computer.

I will not go into detail about the giant maelstrom they created for Pirates of the Caribbean 2, only that it was impossible simulating the whole thing completely. Instead they just used a planar water surface but applied the gravitational and centrifugal forces in different angles so the simulation got it right despite the very shallow water. Clever!

A anecdote Christophe told the audience was about the transformers movie directed by Micheal Bay. The CG-artists did quite a neat job wit the first design for the transformers (that looked much slicker and not as dirty and detailed as the ones seen in the movie). Why? Because Micheal Bay expressed his opinion out the designs. I suggest you make it your desktop wallpaper, so you always stay as motivated as me.

I don’t know exactly about how ILM pre-rendered most of the lighting data so the lighters could work because the rendering times even for previews were just extraordinary, because I simply fell asleep. Oh well… maybe tomorrow will be a better day.

What remains as item on a list of impressions:

Schlossplatz Fountain,
originally uploaded by Phil Strahl.

• A long chat with Martina at Starbucks, trying to cheer her up after she showed her reel to Aardman.
• An odyssey for the cute horse and some black tea
• A stroll though nightly Stuttgart, the Ratatouille credits and a short trip to the fmx party.
• That I lost my beloved Pixar poster somewhere along the way that night.
• No warm water for taking a shower and my everyday insomnia.

What I have learned today

• That I look like some Formula One pilot
• That sitting in the front row makes people feel like they know you from somewhere
• That my demoreel should have the final shot first and then the breakdown when dealing with invisible visual effects
• That applying there at the end of May is a good time
• That every three months I should let them know where I am, what I am doing and send them an updated demo reel
• That sometimes a back-to-the-roots approach can help you with things that are digitally hard to achieve
• That I have no fear of talking into the microphone asking a stupid question in questionable English in front of a very large audience.
• What Dan Curry looks like.
• That a long history of small conversations (e.g. at conventions) can get you the job
• That a crapload of recommendation letters helps getting a visa; big movie names do too
• That only your best stuff should be in your reel, even if you feel it’s too short.
• That you can trust no image, even if moving
• That including easter eggs in your invisible VFX compositing can be used to prove that they arent reality
• That you can trust no image, even if moving
• That something like too much cream in a beverage can exist
• That RenderMan is one fine ass renderer
• That shooting “dirty” footage (= with live action special effects) is harder to handle in the post-pro but conveys a much more realistic feeling to the VFX.
• What Maultaschen and Saiten are in Stuttgart: Swabian maccheroni and wiener sausages.
• That I am too nice which I can’t help.
• That I am too shy in some respect which I can’t help.
• That I definitely feel too old for parties
• That Stuttgart is the capital of traffic lights. Traffic lights that never sleep.
• That writing blog entries until 2:32am just isn’t the best exchange for sleep.

The Plan

Autobahn

Donsch & Martina enjoying Stuttgart,
originally uploaded by Phil Strahl.

Cruising around Salzburg’s Lehen district we found Martina wandering around looking for us and picked her up, still a little over the top from her weekend project:

She and seven other people from class had used the weekend to produce a claymation short film for the instant36 film festival where participants get 36 hours to make a film. That insane effort left everybody involved about not more than two hours of sleep the whole weekend.

Driving on the autobahn to Stuttgart was fast till Munich, following a much slower travel speed because of the tempo limits that made driving on fresh black asphalt a test for your strength of will to restrain yourself from hitting the gas pedal. Donsch and I were discussing effects of radiation on the human body and the nukular power plants along the way while Martina took a nap before we entered Stuttgart city limits at around 8:50am.

Stuttgart is a beautiful city, hid from the autobahn in a valley combining the flair of Mediterranean villages with its houses along steep hillsides facing vineyards; Munich’s cleanness and neatness and even a touch of San Francisco when you roll down the hill on a windy road with rail tracks in the middle.

Thanks to Navigon we reached the Haus der Wirtschaft shortly after 9am and I headed straight for a parking place in what’s probably Germany’s most expensive parking garage: today cost me twenty Euros. Painful especially because I’ve learned from Moritz, a fellow student and native of Augsburg, that there was some kind of student parking lot where they charge you only 2.50 € for the day. Ouch! Maybe tomorrow…

Until our little group of three got their tickets it already was 10am and the show had started. Martina decided to visit the Aardman recruiting session while I was eager to see Euphoria, a software for simulating human behavior at runtime, in GTA IV in action, explained by one of its developers, Torsten Reil from Naturalmotion.

Games and Korea

A break at Starbucks,
originally uploaded by Phil Strahl.

In the short break I met Ivana and Marion from the 4th semester who were so matching in their preppyness that you couldn’t help smiling. I was lazy and tired and wanted to move as little as possible and so I stayed along with the girls for Advancements in High Quality Rendering where Wil Braithwaite of nVidia laid out a workflow for near realtime rendering using heavily the GPU. Obviously we weren’t the only ones who once were raging about why it takes so long to render one crappy frame with MentalRay while there’s so much neary photorealistic stuff going on in the next games in frickin’ real time.

Haarm-Pieter Duiker from Digital Domain took over the second part of this unit and was showcasing the hyped and until Friday unreleased Wachowsky-flick Speed Racer. It was so colorful and over the top that it really hurt your eyes. Duiker was explaining how they managed to get the most realistic car-shader possible and showed composits from the movie that look so artificial that a simple phong-shader would’ve done the job equally well in my opinion.

After a break I let myself talk into Public Game Funding in Korea, held in a small room by Ken Kyunam Choi from KAIGA, the Korean federal department of gaming. Choi seemed to me like a real nice and even passionate guy but his presentation was as humble as his English skills (not that bad though!). His PowerPoint presentation remembered me a lot of any Korean manual to a piece of hardware: Strange Arial-ish typography, fine dotted stripes in the background impossible to tell whether that was intentional, too much text and too many numbers filling each slide and the charming overuse of diagrams. Don’t get me wrong, it wasn’t uninteresting, still I had to fight though this hour when some of the audience kept asking the poor guy the same questions over and over again he just couldn’t answer properly. But I’ve learned that the Korean government sees games as part of culture, provides gaming addiction prevention centers and educates teachers and parents on how to play games with children. That’s totally different than the local policy in Austria regarding games…

In the break the three of us located a Starbucks nearby where I guzzled a venti caramel macchiato which really put me back to life along with some mango-chicken wraps. I even considered purchasing a sitar because right next to the coffeeshop was a store selling all kinds of instruments.

Already history

The Haus der Wirtschaft,
originally uploaded by Phil Strahl.

I really regret that we we returned twenty minutes late for the following presentation, 30 Years in CG by a passionate and very entertaining Glenn Entis, who founded Pacific Data Images in the early 80s. He switched to EA before PDI was swallowed by DreamWorks and made stuff like the Shrek series. And recently Glenn quit his job at EA because he wanted to try something new. I like that attitude!

Alex Laurant continued afterwards talking about his transition from drawing T-shirt designs in the 70s, to illustrating editorials, studying typography, getting involved with multimedia in the late 1990s, then switching to ILM where he supervised the VFX for Minority Report, and now he’s working for Lucas Arts, in more or less the same building. Unfortunately he had to cut quite a bit of his presentation but I was able to get some hints and tips for my current freelance work on Frontiers for GoldExtra.

The Fusion of Technology and Cinematography by Scott Singer was mainly about the underlying pipeline of DreamWorks and when you should buy those guys from R&D some beer ¹. I tired to hunt Scott down after his lecture in order to inject one of my many portfolios but there are always people from India who get there first and keep chatting and fooling around until the next presentation starts and the target person leaves – better luck next time.

Highly anticipated and extensively cheered was Sharon Calahan‘s lighting and cinematography work for Pixar’s Ratatouille. Even though my eyes began to close now and then at random I learned a lot and got inspired for my upcoming compositings, like the way you make fruit and food look appealing.

2D or not 2D, this is the question

With Martina, who also had troubles keeping herself from falling asleep and tipping over we both went to the lounge for a strong espresso and missed the fist part of Marlon West‘s demonstration of the new old Goofy: How to make a Goofy short in an all digital process that looks like it was done in the 50′s. Following his presentation he showed the final short film again and I was highly amused because they did a tremendous job at not getting crazy with next-gen compositing. Another time I tried to catch the speaker and this time it worked after waiting for a nerdy guy from Locomotion to finish his schmoozing. I told Marlon how much I enjoyed the new Goofy short and how great it was to me that Disney returned to full feature 2D-animation with The Princess and the Frog ². And I was asking for a job as an intern at Disney – I am as shameless as I am desperate. He suggested me to show up tomorrow at the recruiting table ³ which is better than nothing.

Eye Candy

The last unit today was titled Shelly’s Eye Candy andit kept its promise. Shelly Page of DreamWorks collected all over the world outstanding animation pieces and showed them to the broad audience. There was so many great stuff in the 45 minutes that I relly hope I remember everything:

• Framestore CFC’s music clip of singing fish, dancing sea horses (I love horsies!) and a beat-boxing fugu.
• Gunniess tipping point commercial, where a rural town has its own kind of domino day. It’s addictive to watch and the payoff really pays off
• Another Gunniess commercial with yellow men being fired at huge drums and falling down huge guitar strings only to dissolve in yellow particles. Nice to watch but the slogan “It’s alive inside!” along with the thought of dissolving yellow men in your beer is a little… tasteless.
• The Pearce Sisters, a dark and brutal, yet dear and funny animation about two ugly sisters who are fishermen. Great style, interesting story!
• Rockband, the intro for the game of the same name, featuring Deep Purple’s Highway Star with some near photo realistic and some stunning animation and rendering. Short but funny. And you’ll love to watch it over and over again.
• The Man with the Chicken Head (or something like this, only French), is a very surreal and very freaky animation about a chicken in a world full of identic looking men who like to go to the Jazz club and listen to some triangle solo. Weird. Hence funny.
• HUGH, some French animation mix between 3D and 2D after an Apache traditional story. Well executed but it was heavy on (French) dialogue without subtitles. It was something about the sky (le ciel) but nothing that spectacular.
• Blind Spot, a funny animation of a guy robbing a little shop, accidentally shooting people and getting away with it. But it’s the “why” and “how” that really sells it.
• Camera Obscura, an obscure (haha) and simply breathtaking French b/w-styled animation. If you have the possibility then watch it!
• The Cold Rush. A headhunter finds the frozen corpse of his target in the snowy mountains and tries to take it down so he can get the reward. Funny as Fargo because of the tragedy.
• International Trickfilm Festival. A Pixar-styled car crash in slowest motion with a crazy colorful jumping horse called Trixie. It’s so weird that you won’t keep that giggle hidden inside you to yourself.
• Oktapodi. Two octopussies (I love this inappropriate plural) in love in a small Italian town are separated by the fishmonger and try to save each other. Fast-paced action and humor – I loved it!

I hope that’s all of them.

What have I learned today?

Dusk at the Hotel,
originally uploaded by Phil Strahl.

• That I know now where the local Starbucks is located.
• Be quick or forget it.
• That some women, if together, spend more time on the toilet than a man can possibly evaluate.
• That some other women spend a fraction of that time on the toilet.
• That games (usually) just play back interactively something that had been recorded in some ways. Euphoria simulates in runtime.
• That doing invisible visual effects is like cleaning somebody’s dirty dishes.
• That color scripting also makes sense in games.
• That the HUD in games shouldn’t clash but also shouldn’t completely merge with a level’s color toning.
• That doing much more by doing much less is key.
• That organic details (like flowers, seeds, stuff like that) add a great sense of realism.
• That it’s the information you give the audience is important, not the details.
• That you may let things go completely dark in compositing.
• That playing with what you should do and what you shouldn’t do in terms of lighting and layouting sometimes is worth it, rather than separating everything visually.
• That sometimes exposing for the shadows is more interesting than exposing for direct light.
• That every room in Germany seems to be cooler (in a sense of a chilly icy blizzard, not in a sense of hip trendy style) than my room in Salzburg.
• That almost every clock works in realtime.
• That coffee still is my friend.