Pipelines of War

Greg Mitchell ,
originally uploaded by Phil Strahl.

The day started even louder than the fat guy jumping down the stairs above my room at 6:45am: With Gears of War 2 (GoW2) and how Epic Games thought up streamlined their production pipeline for those. Greg Mitchell a big guy, well presenter and Cinematics Director at Epic worked twelve years in television before he switched gears (pun intended) and went into the game industry. He already worked on the cinematics of the first Gears of War (GoW) but wasn’t quite 100% happy with the outcome: Not all was motion captured and so sometimes the animation data had to be sped up or slowed down; e.g. a character walks with 70% speed of the captured motion but talks normal, it just looks weird.
So Epic Greg set himself the task of making everything better than in GoW, to stick to a consistent filmic ¹ style and look.
In GoW there was no pipeline, everything was done by the game artists more or less parallel to their tasks within the game. It worked, yes, but it could have been much better.

Greg laid out first of what the needed pipeline need to consist of in order to determine the scene scope and the needed assets. In the beginning there are audio plays (“radio plays”) and animatics for each scene. While art and level assets are being created the mo-cap recording starts with constant input from set- and level designers (e.g. with blocking diagrams) to give director and actors information about the environment the characters are in. On GoW2 Greg worked with real actors instead of having people “to pull away from their desks”, made enough rehersals before the capture and played back the edited soundtrack with the voice actors on set. All that led to a much higher and better quality and the production speed improved significantly.

After the recording the layouting process starts where the scene takes shape, gets a pace and the cameras are set. Although one might think that motion-captured data is pretty rigid to work with it is not. To get better angles for over-the-shoulder-shots the characters can change their positions a bit or some parts of the animation can be repeated between the shots, e.g. to use a walking sequence twice in succession to give the impression of a greater distance.

This reminds me of what I’ve learned from my mentor Fraser McLean who always told us that we can cheat if it helps the story. He’s currently working on a book about the history and role of layout in traditional animation vs. computer animation today. Because layouting is the most important step in every production I think you should get it once it’s out.

Once the layout gets locked, lighting and effects artists add atmosphere and mood to the scene while the audio department populates the soundtrack with effects and music that is specifically composed for key scenes. You need to bring in the game-designer(s) and producer(s) into the feedback loop as early as possible to keep the revisions to a minimum. Needless to say that there is a constant bugfixing and polishing going on; “With cinematics you’re never done. Never. But at one point you just have to say that it’s finished.” Greg ended.

The art behind making a game

Matt Aldrich,
originally uploaded by Phil Strahl.

Matt Aldrich, the Art Director of Lucasfilm Animation in Singapore, shot up a big image of the package design of the game he was working on. Star Wars Clone Wars for the Nintendo DS. It is sacrilegious to say that you’re sick of Star Wars? I just know that I really am. Nevertheless I tried to be as unbiased and open as possible. First off Matt showed excerpts from the design document which was really thorough and had everything plotted out very detailed. The level design was then outlined in Google SketchUp to visualize not only the key areas but also the players progression through a level. After approval of this very rough layout the art process is started: Based on the detail-lacking SketchUp renderings storyboards are drawn from the player’s perspective int he level and the key moments he or she’ll experience. This is mainly to point out issues before it is costly in time and hence money to make the necessary changes. So take your storyboards and discuss them with every member of the team. The engineering-guys for example will be interested in the amount of polygons simultaneously on screen, the character designers in how close we see the enemies and so on.

The next step was defining a color arc of the scenes in the level. In the level Matt was showing there was a progression from cold, steel-blue colors in the beginning to hot, orange colors at the level-boss fight scene. While it is possible to spend quite a long time drawing those, it is much faster to find reference images and color correct them in Photoshop or do quick paint overs until the colors are final. The focus also laid on key areas for these pictures because nobody has neither time nor manpower to have detailed concept art and paintings for all areas of a game. Once camera angle and perspective where also laid out and locked, concept paintings could bring in the color from the references and add details. In fact those images where so big, that they also became a source for textures for the game. The concept paintings also showed whether it was for the player easy to progress quickly enough through the level. And again, concept art is there to open the discussion and to make it easier to be specific: “The pylon on the left should point in the direction of the player’s goal” is much better than “give it a slant and make it look good”.

At any point it is important to always recall the limitations of the target system. The NDS has very small screens (256×192 pixels), the texture memory is very limited (so the use of vertex lighting was quite important and extensive) and it is a device you can carry around and play in every light situation. So the art department had to focus on good contrasts, very legible silhouettes and a clear level design.

It is incredible what those guys in Singapore did on the DS: The characters for the in-game cinematics have a quite sophisticated animation rig, so they can show facial expressions and talk in lip-sync. I was shocked and awed. But in a good way. Matt went on with how they expanded the Star Wars universe and developed parallel to the TV show for new planets and space ships. But let’s be honest guys: It looks pretty much like any fantastic sci-fi stuff, like all Orcs and Elves and Goblins look alike throughout the fantasy-genre. So I didn’t take any more notes in this presentation. I only know that I want to give some DS homebrew stuff a chance.

Nuke ‘em

I switched rooms and went to the heavily crowded lecture ambiguously titled “Stereo-3D Film Post Tools and Algorithms which turned out to be a presentation of what’s hot and steamy and in beta in Nuke 5.2 by at The Foundry. Surprisingly I got a seat in the second row and had a good view on Simon Robinson’s presentation. In fact it was all about fixing terribly shot stereo ². Simon, head of development at The Foundry, really knew what he was talking about and showed all tricks in Nuke rather than just running a PowerPoint visual hell presentation.

First of all he outlined how to work in Nuke with stereo imagery. You either can use the JoinView node on top of your tree after reading the different eyes ³ or have them already combined in a single EXR file. I asked Simon afterward which one was faster, but he said that it would depend on where you read your EXRs from, what type of CPU you use etc. So I make a wild guess and say that there’s practically no difference. Simon went on to tackle specific stereo problems that can occur in live-action shoots. “If everything was shot right in the first place, none of us would be in this room.” Well spoken.

• The O_VerticalAligner node can compensate for incorrect image alignment but obviously can’t deal if there’s a shift in parallaxes because of it.
• O_ColorMatch is another node that helps to match the images of the stereo-cameras together. Color discrepancies often occur when one eye was shot through a mirror in order to get a closer interocular distance ⁴. While this node does not a perfect job, it does a rather well job and makes it a lot better.
• Nuke can calculate a disparity ⁵-map from the two eyes via O_DisparityGenerator. The stringer the color, the stronger the disparity is. Currently this flickered a lot but “see me in a presentation in a couple of months and this will be much better”. They’re always improving.
• There’s also the possibility of setting a convergence point. The advantage is, that it can be done for any pixel in the image, so dragging a convergence point over a moving object can keep the focus on it.

At this point Simon switched to the anaglyph view in Nuke and I got a little upset. When entering, people were given a set of paper-polarization filters that have a distinct gray color ⁶. So on the appearance of a blue/red anagylph image on screen about 95% percent of the people I saw around me put on their filters, without even thinking about it. You just won’t see a 3d image in an blue/red anaglyph-image when wearing pol-filters on your nose. Some folks kept them on for as long as 30 seconds before realizing it. Sheesh! And here comes the kicker: Most of these people did it a second time with the next anaglyph image just a couple of minutes later. Some people just drive me nuts!

• A clever and time saving idea is the ReConverge node that pushes everything the artists did from one eye to the other, e.g. roto or paint. “It won’t match perfectly, still puts you more than half the way through. You only have to tweak it instead of recreate it.”
• O_InterocularShifter comes in handy when the interocular separation between the two eyes was shot too wide and you have to fix it. This node calculates a new set of stereo-cameras that are positioned between the original ones. Currently it took Simon’s notebook about 20 seconds to calculate a frame. “It’ll be faster next time” he promised. Still it is nothing to correct an entire movie with because there will be occluded objects in the right eye and occluded object in the left eye which can’t be magically thought up by the software, the disparity estimation won’t work then. So it’s more a tool of last means rather than a way of remastering your stereo IMAX movie for television. However it can assist CG pipelines.

So does stereo just keep making our lives harder? Yes, still some things work better. Like camera tracking because you have way more depth information which in turn results in a much more stable disparity map. from that you can pull a Z-map of your scene and add things like volumetric fog in post-production or correctly pulling the digital lens for some depth-of-field-effects. Another thing that will be coming along is that more and more metadata from the shoot will be used in the compositing process, eventually even autmated. Until now we had the pleasure of running around with clipboards, tape measures and constantly bugged the DOPs. At least I know I had.

What’s still in development are things like lens-distortions or how to deal with optical effects such as lens flares or blooms in stereo. What I have learned from last year is that you either have them in either both eyes or no eye. Further Simon talked a little about using more than two cameras to get even more information form a live scene, “The algorithms are there.”

Essentially this lecture was was a down-to-earth showcasing of The Foundry’s Ocula Plug-In set. If you want to bug the poor man even some more: Here’s his address sam@thefoundry.com.

Small is Beautiful

Richard Hilleman ,
originally uploaded by Phil Strahl.

After the break gaming veteran Richard Hilleman from Electronic Arts held an inspiring lecture about the evolution in games. We all knew that in the early 1980s pretty much everybody with a computer, programming skills and a good idea could make a game and, eventually a lot of money. Fast forward 25 years: Today there are a handful of big players and about 50 teams (worldwide) that can pull off an AAA high-def game costing 25 million dollars ⁷. While all these big studies can’t afford to take any risks and have to please a very broad audience, the small designer can do whatever she or he wants because the stakes a lot lower. If four people get together, work on a browser game for a couple of months that did cost them, say $500 in total, and they make a profit of $10,000 that’s a huge profit margin. Yet $10,000 wouldn’t probably even cover EA’s monthly coffee bill.

So how do you make a great product then? It has much to do with yourself:

• Show passion not only in making the game but also for its content. Richard Hilleman has a passion for Football. So he created the first Madden NHL which was, as we all know, a success.
• Be versatile! Nobody’s going to hire the 8th shader engineer. But if you are a shader engineer who knows how to manage a group of people, about their tasks and see the bigger picture, your chances will improve drastically.
• So learn more than your base skill and get technical as well as leadership experience. You’ll learn much more when you have to lead a team that’s so big that you can’t do what’s missing in the end yourself.
• Be curious. Explore. Obtain knowledge. “Don’t accept the box they try to put you in”.
• Learn about money and how it works with your product. From start to finish. Internalize it. Understand the economics of your product. There’s just no way around it.
• Learn people. Because “Everything you learn technically will be gone in 7 to 10 years”.
  • People are your customers.
  • People are your team mates.
  • People are your means of expression.
  • People are you inspiration.

  Change people’s minds. Surprise them. Take them on a journey. Entertain them!

You won’t need a huge target group. The target group for games usually is between 14 and 20 years old and male. They have the time, their parents have the money. But you can’t experiment much inside that target group. On the other hand there’s Pogo where the average person plays for about 20 hours per week. “Would they consider themselves as gamers? No.”. This market for casual games is evolving. There still will be the audience for high-def games but don’t forget the long tail. Pogo’s average customer are 49yr old women, for example.

“Who do you rather want to be: Michael Bay who gets $250 million to shoot some producer’s movie or Robert Rodriguez with $5,000 shooting his own movie?”

The Tale of Framestore

Andy Lomas of Framestore CFC told the tale of an English post-production house that was famous for its commercials when it set itself the task of creating an animated feature film, namely The Tale of Derspereaux. The most interesting part of it was the way they mimicked the lighting on paintings from the old masters Bruegel, Vermeer but also Bosch; that they had to use mouse-scale cameras for the proper depth-of-field effects, used filmic dollies and technocranes and made the image deliberately imperfect by blocking the view or some jitter here and there, have even more flaring and blooming and so on. Nothing new, in fact. I’ll cover the cinematography of WALL·E below.
Personally this lecture didn’t intrigue me much. Yes, Framestore showed that they can pull off making a full CG movie in Europe by themselves but there was nothing striking to me. In my opinion even the look wasn’t that top notch but still waaay better than that horrible Back to Gaya.
Andy showed their tools such as asset and production data management and so on. He talked about the shift from an asset based workflow (“what stuff is needed?”) to a shot based workshot (“what’s the story here?”). Also Andy stressed the importance of layout and previz (nothing new, huh?) as means of a creative hub, bringing the costs under control and to lock down as much as possible as early as possible. Again, bring in the clients as early as possible in the feedback loop for their involvement is essential. In fact is the final feature nothing more but a very refined version of the layout.
In the end he showed some production tools Framestore had used such as their production asset management tool Shotgun or Pick a Prop that linked the Object ID pass in an EXR to the asset database and displays the name of the prop the pointer is hovering over. This was mainly to ensure a clear communication such as “What’s wrong with the shadow of cr_p_wooden_barrel_v54″ as opposed to “What’s wrong with the shadow of that brown thing in the background?”.

Let there be light

Danielle Feinberg,
originally uploaded by Phil Strahl.

What followed was the lecture most of us were waiting for the whole week (the fmx folks really do save the best for last). Danielle Feinberg, who was one of the Pixar DOPs on Wall·E, explained how and why the feature got its distinctive look. As I already knew from last year’s Pixar presentation they really like to research extensively and try things out for themselves. And for Wall·E they found that their lighting model and camera code was outdated. Director Andrew Stanton’s vision was to show the abandoned earth through the lens of an 1970′s science-fiction-feature camera, with all the distortions and funny stuff going on. To test things out they filmed in the atrium live-scale models of Wall·E and Eve with test patterns all around and a grid on the floor on 70mm stock and with anamorphic lenses. Hey, they even got Dennis Muren to show them the ropes! So according to their tests their camera-code was adjusted. Also, they set themselves the limitations of having only a certain set of lenses (see box). Now Pixar operates a fully functional virtual 70mm camera with anamorphic lenses and all the artifacts that they bring (optical breathing, barrel distortions, lens flares with blue streaks, elliptic highlights and so on). If you don’t overdo it you get yourself a look.

To develop a look the folks at Pixar also researched extensively and came down to that 1970′s science-fiction-feature look. Orange, documentary, existing light is used and, just like Sharon Callahan said last year about Ratatouille, don’t be afraid of the dark i.e. let things go to complete darkness if it is justified. But again, Pixar failed on that. I guess they tend in general to over-light their features in some respect.

For the shading they came up with a new illumination model of energy conservation ⁸ that essentially comes down to three knobs: reflection, specularity and roughness. Basically the rougher a surface gets the less it reflects and the amount of reflected light is never higher than the light the surface received. The new shaders also are capable of ‘hot reflections’ and perform realistic fesnel falloffs themselves in the rendering.

The shading of Eve was much more complicated than anticipated because she is made up so many parts that should fit together seamlessly, yet has circuits and light on the inside and goes through quite a lot of transformations. On the other hand Wall·E’s eyes were also an important part for his performance so he wouldn’t look dead (too reflective eyes) or creepy (too little reflecting eyes). He got his final appearance by lighting the aperture blades inside so they would break out visually from the blackness of his eyes.

Because I was more concerned with the technical side of this lecture I don’t have any notes taken on the other topics that were touched, but I bet there’s an artbook already out where you see many of the beautiful drawings, silhouette and color studies and so on. Pixar artbooks are either way and obligatory possession and resource, even if you’re only on the outer rims of the industry.

Waltz with Michael

Michael Faust,
originally uploaded by Phil Strahl.

The lecture was just called “Waltz with Bashir” and the only thing I knew about the film was a 5-second clip I had seen many months ago which made me eager to see it. Unfortunately I missed seeing the film once again. But not this lecture featuring the stunning look the Bridgit Folman Gilm Gang hat achieved. With Adobe Flash! Michael explained that it was the first feature film of The Gang, nobody had worked on something that big before. In the course of the pitching they animated a scene using the cut-out technique in Flash by separating body parts and just moving them around until a new keyframe was needed to be drawn. At first the segments were rather large and soon they figured out that it wouldn’t look good enough. So more and more shapes were broken into their components until a face was nothing more than a flesh-colored blob with dozens of tiny black snippets that the animator used to animate the face.

The backgrounds often came from photo references that had been traced and painted over in Photoshop; some elements were completely thought up and yet they integrated perfectly into the realistic environment. Michael, who worked as an Illustrator and did some backgrounds said that it was hard for him at first to change his style he was used to from his oil painting to something so completely different.

When the animation work on the film could begin, the director already had the film finished, leaving out lots of black holes and studio reinactments of what needed to be animated in the final film. So the layout phase began. It was done mostly traditional and very sketchy with indicators of what needed to be a new keyframe that had to be drawn. The characters and poses were drawn by hand and then in Flash painted over, the backgrounds traced in Photoshop to match the very narrow color palette. Any effects had been done in After Effects such as trails of smoke. Michael brought some animatics for us to view (it still was odd watching him open .swf files of what ended up on real film stock) and they looked pretty much like uncleaned finals, their quality was just outstanding. Like the rest of the film. Because of the tight budget there was no room for motion capturing or painting every frame by hand. So this Flash-based cut-out technique, as tedious as it may seem, was still faster and cheaper than traditional animation.

What party?

There it was again: My lack of sleep kicked in hard and so I decided against watching the animations from the SIGGRAPH Asia which I regret bitterly. I went back to the hotel, slept some hours and woke up just in time to visit the closing party. But you know me: I don’t like parties because there’s nothing for me to enjoy: People are drunk and pushy, music is too loud to converse properly (also there’s not much to discuss with drunks) and the only people you meet are party people. So I stood in the hotel and tried to catch up some sleep for the journey home. I failed.

After enjoying the breakfast a little too long I was rushing down Königsstraße in my car so I would make it to Pixar’s RenderMan presentation. I already knew what it was going to be considering last year (“The Über-Sprite”, the rocket, the fast-rendering motion blur) but Pixar is rather generous in handing out posters and presents and I wanted me to get another teapot for my collection ¹. I was too late, the room bursting with people. Obviously, word had spread that you get presents. People can be so greedy. I asked if I could make a reservation for the afternoon but it was in vain.

There I was standing, lacking a teapot and a clue of what I wanted to see instead. I headed to the biggest hall and ended up in “PhotoReal Facial Animation” by Patrick Davenport and Steve Caulkin of Image Metrics. They showed the sample clips I already knew so it was no surprise to me that… (click “show” to view spoiler) [spoiler]…Emily’s head was CG.[/spoiler]

You can find the clips also at YouTube if you’re interested.

The crazy stuff Imagemetrics does is providing face tracking with only the use of a video camera. The tracked regions of the face are then moved on a CG model. Tweak the keyframes and you’re done. It’s that easy! Well, it’s not. Steve Caulkin laid out the long way to their Emily demo which occurred to me as not really time saving: Apart from photographing the actress’s face for the texture, there also had to make a cast of her teeth but the molded teeth wouldn’t necessarily fit correctly so you end up taking x-rays to learn how to place the teeth correctly. And that’s only the beginning.
When scanning the different expressions of the actress the data was anything but coherent so somebody had to clean up all the meshes (about 55) and get the details out: Pores and such can only be done with a bump or displacement map. It would be just too much for the statistics-based tracking algorithm.

Steve Caulkin owes me a venti Caramel Macchiato. His presentation was in-depth and very interesting but, alas, Steve is more a guy you put in front of a C++ compiler than in front of an audience and it was hard to follow his low pace.

So I ended up at Starbucks with an iced caramel macchiato before making another attempt at getting into one of Pixar’s presentations. I queued up 20 mins and before they opened the doors there was already not much oxygen left. And I felt the urge for another caramel macchiato.

Pixar’s Carreer Gears was a again a valuable information on how to apply and how to put your reel together for Pixar. Right in the beginning the panelists ² asked the audience to raise their hands of what position at Pixar they’re interested in. To sum things up: Two thirds were character animators, many wanted to become story artists and only a few people were interested in the other stuff. And I bet I was the only compositor in the whole room. Here’s what’s strange: Last year I was told that Pixar doesn’t really do compositing which I thought was a joke or they wanted to pull my leg. Today they also didn’t say anything about job openings or positions in compositing. Very strange.

The panelists talked about their experiences at Pixar and how they got their job and spread the usual tales of people who were hired right off the college. Then they took questions. I must have dozed off somewhere in between but it was mostly asked on the process of applying and what Pixar is looking for. Here’s the stuff I remember:

• Don’t send in a reel when you have nothing to show.
• Put your name on everything.
• Have the DVD region-code free and tested to play on a standard set-top DVD player (NTSC and PAL both are fine).
• Apply for a certain job instead of just applying for the database.
• Send every 8 to 12 months an updated reel to show how you progressed.
• Don’t send every week new reels.
• Write a decent cover letter. They’ll read them.
• Don’t forget the all-important shot-breakdown. Preferably even on screen.
• Don’t chase the ostriches on the front lawn (I guess that’s where I dozed off).

Interestingly I gradually lost my interest and my caffeine addiction kicked in hard after an hour so I left for a chili dog and a precious cup of coffee. At Starbucks they either love me or hate me.

For lack of motivation to look for the right screening room for “Analog Artifacts in CGI” I went with the crowd to witness “Skin & Lighting Research” by Christophe Héry of ILM whom I already know from last year.

Holy moly! In his presentation I saw more formulas than in my whole college education ³ In fact he told nothing new about subdermal distribution and the models on how to calculate them (although I didn’t understand much of the math). So far, so good. But what If you can’t afford raytracing because, say, your artists produced more vertices than the final rendering will have pixels (see Pirates of the Caribbean II: Dead Man’s Chest)? You’ll have to have an point cloud based approach to dodge memory demanding raytracing. And when you don’t have raytracing going on RenderMan really does the trick fast and good.

Cute as a Button

That lecture served as the perfect introduction to what we all have been waiting for: The Curious Case of Photoreal Head Replacement.

Jonathan Litt had a huge presentation explaining the lighting, rendering and compositing of that huge task. How do you start? They started with a artfully crafted latex-maquette of Brad Pitt’s face made old, for it had a really realistic appeal in subsurface scattering and served as most valuable reference when comparing renderings of the CG head to it.

The head itself was done in Mudbox (yay!) and in it’s highest resolution had about 4.5 million polygons. This high level of detail was preserved by using displacement maps, that further were driven by curves so wrinkles would get stronger or weaker depending on the facial expression. The eyes were modeled and textured anatomically correct (I’ll just throw some expressions at you of what they considered: caruncle, meniscus, conjunctiva, sclera, cornea). As further reference they had a extreme-high-res photograph of Brad Bitt that you could see the micro-wrinkles between his pores. “That’s thousand dollar pores!” Jon joked.

But this perfect model also needed to be lit in perfect coherence to the on-set instruments and light sources. So additionally to the high res long-lat-HDRs that were taken on set, there were extensive survey data on each shot of all the light sources and scene geometry so that the HDRI could be mapped back in Maya onto this surveyed geometry.

The maquette of the head was photographed in LightStage with light from all possible directions (separately). A script then made it possible to color and blend these separate light-passes together based on the information of the on-set HDRIs. Why the hassle? Because the renderings were put next to this near perfect reference and the artists could check on how close they got.

The next obstacle was to choose the right approach on how the HDR sampling should be done, either Inside-Out (I-O) or Outside-In (O-I) from the HDR. The I-O approach is usually used to sample the environment for Global Illumination. You have to fire a lot of rays to cover correctly bright light sources. So you need to find hot spots and treat them as emissions. I-O works well with spheres but with other geometry you get shadow bending ⁴. The solution to this problem was to scatter the origin of the emission-positions during rendering (see the paper of Kollig & Keller, 2003).
probably guess that it didn’t simplify things that the head was moving through the scene.

The solution to all this blocking and head-movement was to reposition the HRDIs on every frame on the position of the body-double’s head. Because there was enough tracking data of the head moving through the scene the mapped HDRI in Maya was rendered in Nuke to match the position of the head which was much easier than doing it from scratch.

What comes now is really sexy: To single out light sources the direct practicals and instruments visible in the HDR were blocked or painted out in Nuke resulting in an HDR image of the ambient lighting. The missing “hero lights” were then positioned as area lights in Maya and given a HDRI texture. This was also very important for the eye-lights.

Still there had to be adjustments made for the eye sockets and eye-lights because on set the lighting was done on the body actors.

I really realized that I want to work at Digital Domain: They value Maya, Mental Ray and, most important Nuke. Adopt me!

Blogging,
originally uploaded by Phil Strahl.

The last presentation I saw before going to my car for some sleep was by Steve Preeg on the Animation and Performance of Benjamin Button. The big issue was on how to capture the performance of Brad Pit and have it applied to the digital head. And because the show was a $ 150 million Fincher/Pitt movie there was no room for error. If you’d mess it up, they would mess you up.
To get all the muscles in Brad Pitt’s face right Preeg thought about CAT scanning him but his manager just told Steve to think of something different. And so he did. Initially Digital Domain got the guys from Mobile who had developed a volumetric capturing system and captured various key poses of Brad Pitts face as basis for the blend shapes in Maya. When everything was tested and worked on they needed to capture the actual performance by Brad for the digital head.
They had him watch the clips from the movie with the body actor so he knew what was going on around the him. During his performance his face was filmed from four different positions, his cues were given brad via in-ear monitoring. In fact, Digital Domain even tried Imagemetrics but the result was too ‘dead’ to them, however it helped much in timing the animation which was all done by hand. Thus it was guaranteed to keep the intent of the performance rather than applying it with strange results. “Sometimes is just a millimeter more or less on one of the eyelids between creepy and cute”.

What I have learned today:

• That Steve from Imagemetrics probably wouldn’t pass a Turing test.
• That you can capture the facial performance of actors during motion capture by having them wear head-mounted camera-rigs with a light source both pointed at their faces.
• That on a Z-buffer approach to subsurface scattering the resolution of the buffers matters a lot (bigger = better).
• That on a Z-buffer approach to subsurface scattering you should keep the buffers separate, meaning that nothing that’s not part of the skin may cast shadows inside the skin.
• That on a Z-buffer approach to subsurface scattering won’t let you have your precious raytracing. So nobody does it anymore.
• That you best take texture photographs of skin by having polarization filters on your lights and one (90° out of phase) on your camera. Thus you block out the specular highlights and only get the diffuse light. Still you need to paint out shadows. Use 6 soft lights when you don’t have the luxury of having a Light Stage.
• A big deal in believability in CG skin are oil layer and peach fuzz. If you can’t nail it down why something doesn’t feel right then it’s usually one of those things.
• That working on 64 bit machines with 16 gigs of RAM really saved Digital Domain’s ass in producing Benjamin Button.
• That the UV-Space in Nuke (if kept in the EXRs) can save much time for last minute changes on textures.

What surprised me today:

• That you can talk passionately about human emotion without showing any.
• That Image Metrics also use Eurostile as their house font. Like me. And they’re not the only ones so I really should think of a new font then…
• That relatively few people who want to work at Pixar are interested in lighting, shading, layouting, rendering, controlling, software engineering or cinematography. They all want to become animators, character designers or, cough, directors.
• That I used working with z-buffered renders a lot in the hey-days of the late 90′s. I feel old.
• That not a single CG spotlight was used for the lighting of Benjamin Button.
• That Brad Pitt’s teeth were too white to pass as a 70-year old. For the digital head Steve Preegs teeth-color was used. That’s why he quit smoking on the show.
• That it was the first time that I read ‘LOL’ in a presentation. It is 2009 and netspeak finally conquers offline-speech.

Along with some peers we came just in time to the Metropol theater where the screening of the stop-motion adaption of Neil Gaiman’s book Coraline was rolling. In stereo. The film remained quite close to the book and featured some very inspired and inspiring designs (keep your eyes open in the garden scene — lovely!). In my opinion the film still got a little too American but hey, it’s a big production after all. The animators did a tremendous job: The cat really moved like a cat and Coraline was most convincingly animated in the top-shot when she creeps into her parents’ empty bed. Further I’m thinking about buying the soundtrack. So if you consider yourself only a minor Gaiman fan and are not following him on Twitter ¹: Go for it, it doesn’t hurt your brains.

Back at the convention center Chris Williams of Disney was talking about story telling in his short Glago’s Guest that I already knew from yesterday. At least that’s what the schedule said. After seeing the short again Chris showed us the final storyboards first, then what story ideas were thrown away along the way of improving it until it worked. He went on to the designs, the color script and the overall style of the film until he showed it one more time. One thing that still bugged me personally is the action of taking out the garbage because it is such a deep rooted American suburbian tradition that it felt really off in the setting of Siberia in 1924. And the garbage can itself was as American as Uncle Sam on 4th of July reciting the Bill of Rights. I will finish my nitpicking on this one by stating that this lecture didn’t really deal with story telling that much.

Richard Edlund ,
originally uploaded by Phil Strahl.

After the lunch break where I enjoyed a chili hot dog, Eric Roth, chairman of the VFX society talked to VFX legend Richard Edlund about his work in the early days of visual effects in movies such as Star Wars, Raiders of the Lost Ark and Poltergeist and also Die Hard and Ghost. Nowadays when everything is so easy and every 16-year old with a computer can make stunning VFX, one forgets that in those old days visual effects were as complicated as they were time consuming. The imploding house in Poltergeist took an artist eight months to rotoscope. Hell! To my regrets this interesting panel passed way too fast.

Syd Mead,
originally uploaded by Phil Strahl.

Syd Mead, the one and only, held his presentation (the same he gave on the Siggraph Asia before) about his rise from early childhood scribbles to the latest designs. His childhood images already showcased his early fascination with cars. It was incredible to see his futuristic visions of the late 1960′s and early 1970′s with car designs that look familiar with today’s eyes. Syd really knows what he’s doing and has a story for every of his paintings. In a near photorealistic rendering of his Hypervan he points to a chrome-like disk somewhere on the outskirts of the painting “This is the security droid in this marina”. Every painting he showed us had a story and he could talk in detail about every detail. Except for the bathroom design for Blade Runner. “Do you know what this is? I don’t either. It just looks like it belongs in this bathroom.” He has funny explanations for anything, not only in his paintings. “Do you know what Gouache means? It’s French for ‘bitchy medium’.”

Habib Zargapour,
originally uploaded by Phil Strahl.

Habib Zargapour was in the unfortunate position to speak after Syd Mead still he pulled it off quite well. From his experience, coming from films to games, he outlined the similarities and differences between designing for games and designing for movies. Still, a lot of principles are alike, yet the biggest uncertainty factor is that you can’t control the camera, so you have to control the environment and make sure it works from every angle. Further you can’t work on a shot-by-shot basis.

“Visual Storytelling in Computer Graphics” by Harrison Ellenshaw had the charm of one of Fraser McLean’s seminars: He talked passionately about the principles, underlining them by showing clips of great movies including ancient Disney features. The films that he showed to the audience (Ryan’s Daughter, Cinderella, Dr. Strangelove or How I Learned to Stop Worrying and Love the Bomb) he commented with such a passion and enthusiasm that you couldn’t help but feeling it yourself: Wow, movies are the greatest and purest thing mankind has ever produced.

Again, this year Shelly Page from Dreamworks brought the fat of the land (mostly France though) of animation to us in the last hour in her “Shelly’s Eye Candy” presentation. Here’s a complete list of all the presented films:

• Yankee Gal, the moments in the life of a WW II pilot in a crashing airplane.
• French Roast, very funny, very French animation about an unappealing protagonist. I love those kind! And the coughing clochard!
• Steel Life, so very abstract and visually strong, like a remake or homage of Koyaanisqatsi would look like. And, no surprise, the music really drives it home. I mean *really*! Composer was Mathieu Alvado.
• Carver Audi Commercial by Framestore CFC. As usual visually very strong and makes you wonder before the payoff.
• Avatar Coca Cola Commercial, also Framestore CFC. The connection to the product itself was totally random, I guess somebody just loved the idea of populating the world with avatars.
• Stork Monster commercial, another Framestore CFC thingie. Very good idea, very well executed. As always.
• Flap Flap, German short about two ravens. If you’re into toilet humour you’ll laugh. In my opinion: crappy (pun intended). Found no link, sorry!
• They Will Come To Town, as seen yesterday. As impressive as yesterday.
• Guerre Naïve, very French with nanoloop musics and, yes, F-Zero countdown sounds about a racing boy. Strange. The French try to imitate the Japanese and vice versa ² in animation. I guess they have a crush on each other — cute!
• For Sock’s Sake, a Calarts graduation animation about a lost sock and his family of other clothes trying to find him. A very fresh idea and witty, expressive animation.
• Ex-E.T., about an alien child that’s just not in sync with his environment. Very good payoff. You’re gonna like this one (or at least the end).
• Dix, about a neurotic’s torment to overcome his compulsion. Very gory and disturbing at many points. Top notch!

Again, I had troubles keeping my eyes open and my mind sharp but I succeeded. Still I had to spend the rest of the evening in my car, sleeping, before paying an fmx party (‘Echtzeitparty’) event a visit with some of my peers. I really don’t like going out. Today I got reminded of that fact yet again.

What I have learned today:

• That it is impossible to find a parking garage that’s more expensive than the one I use.
• That creative argument is the best you can ask for. Any idea only gets better when creative people keep chewing on it.
• That it often helps to get new ideas by drawing without constantly thinking about what you’re drawing.
• That cuts that don’t cut into action are very in your face. If you want it that way, then have the audio have the same harsh cuts.
• That story is about change.
• That you shouldn’t overdraw your storyboards. Only draw what is necessary to the understanding. Then break that down into the least amount of images possible.
• That chroma keying on a chemical basis is like sumo wrestling: You have this huge opponent and you just want him out of the ring.
• That production wise VFX are a tightrope between the producer and the director.
• That you should trust your instincts once you get better.
• That when you draw people in long robes you don’t have to worry about drawing their feet.
• That you got to have a story behind/in your painting, no matter how unimportant it might seem.
• That you get interesting designs when using cliché for you audience to instantly recognize where you are going to take them, then add a new unusual wave to it.
• That constraints help good design.
• That ‘weenies’ in environment design basically are landmarks: They help you navigate the environment.
• That in first person shooters you tell a story basically by how you lay it out.
• That (in games) “story means action” (Habib Zargarpour).
• That ‘Divided Highways‘ is a good book on architecture and, indirectly on level design. So are ‘Learning from Las Vegas‘ and ‘Carchitecture‘.
• That concept art is what helps people to agree on something — thus saves money and time in the end.
• That television is a sales medium. You want people to get involved so they’ll watch the commercials.
• That you should tell as much as possible visually.
• That “action is character”, it is defining the character(s) — (Harrison Ellenshaw)
• That when you move the camera, everything moves. Does your story really wants you to move everything? If not: Keep the cam rigid.
• That “The Curious Case of Benjamin Button” really is scary stuff for actors, but so was “Cinderella”. But…
• …CGI is just another tool that won’t replace real actors or real humans operating those tools.

What surprised me today:

• That George Lucas is said to be rather introverted. Just like his chin suggests.
• That short films by the big studios don’t make any money. In fact, they only cost the production a lot.
• That all the helicopters in Die Hard were added in post. All of them!
• That the movie “Ryan’s Daughter” has absolutely great pictures.

Hotel Hottmann,
originally uploaded by Phil Strahl.

When roaming the Stuttgart streets again it didn’t feel as nice as last year. It was cold, it was foggy and some drunk junk was shouting profanities and bugging people on the Schloßplatz. A sharp turn took me to Starbucks where a friendly caramel macchiato was waiting for me and where I decided today’s program amidst men in fancy suits and a flock of girls skipping school.

I was among the first few visitors who showed up at 9:30 to view the introductory selection of short films from the Filmakademie Ludwigsburg featuring Urs, Something’s Coming, Lebensader, a short for the Cartoon Forum and finally the long version of the fmx’s visual jingle, Frequency Morphogenesis. Conference chair Thomas Haegele bade us welcome and without a transition the first presentation started: “Previsualizing 9/11″ about the previz process of Oliver Stone’s World Trade Center.

John Scheele and Ron Frankel talked about the long and thorough previz on that film because there was no room for stylization because we all know the disturbing pictures by heart; “Documentary footage becomes the iconic reference of an event”. The previz they created was used by all departments throughout the production phase and was like puzzling together what was happening on a grand scale and what the real survivors experienced. “It was understanding what really happened vs. what the two survivors thought they saw”.

It was not possible to shoot on the real Ground Zero for all the terrible memories the scenes would evoke, so the production needed to pursue a different approach. High resolution HDRI photographs were taken from the surroundings so they could be used to populate the digital recreation of the site. The film was entirely shot in Los Angels, partly on Lebanon Street, the only street that looked somewha Broadway-ish.

The previz was divided into a practical previz for the different departments e.g. what the camera crew needed to know, the set decorators and so on, and into a post-viz meaning where buildings needed to placed correctly after the shoot was done. Ron Frankel re-created a large part of Lower manhattan in XSI up to the details needed for getting the big picture as well as what the survivors saw — they believed until their rescue that a bomb went off in the garage.

My ticket,
originally uploaded by Phil Strahl.

I skipped “A Global Production Pipeline” Xavier Nicolas from Lucasfilm Animation for some more java at Starbucks before returning just in time to Sony Imageworks’ “Animation and VFX” by Bob Osher from Sony Imageworks. His presentation felt at first like its target audience was potential shareholders and the emphasis on “Innovation in Support of the Filmmakers” sounded as cliché as does the slogan “The Future — Now!” ¹. Thing got a little more interesting when the Arnold renderer was briefly touched, although a little too sketchy but what really blew me away was when Bob introduced Sony Imageworks’ inhouse post-production tool Katana ² which I understood as an optimization tool that interconnects 3d and compositing back and forth and saves big amounts of time and, effectively, money. In the course of the presentation I saw a the node tree of the wide shot in Watchmen where Dr. Manhattan blows up, well, Manhattan that was also done with Katana. Speaking as a Nuke compositor I have to admit that it made me kinda frisky.

Generally speaking: Sony has a lot of sophisticated in-house tools to help the artists and is very proud of their upcoming feature Cloudy with a Chance of Meatballs. Bob showed us the Jell-O scene and after 30 seconds he won us all for it. Can’t wait to see it in “mouthwatering 3D”.

Now we all were hungry so I got me a nice hot tomato soup with bread for lunch and roamed the stretching shopping boulevard, eager to find some place where I could by a shaver and some eyeliner.

After the break followed a little panel titled “I Got A Job Abroad… Now What?”
hosted by Jan Sjovall and featuring thee more Germans who made it abroad. The room was already full when I arrived so I was cramped into the back and sat rather uncomfortably close to the floor and the informational value of the panel was scarce. Still they dropped a few things to consider when working abroad like that you only realize in comparison how different your own cultural background is.

Over the day I met some folks of rise fx where I spent last summer four months as part of my internship. It was fun chatting a little and so I decided to see their presentation that was part of a broader presentation of the VFX and animation facilities in Berlin-Brandenburg. I already struggled a little with my sleepiness — four hours definitely are too little.

Watch ‘em

Haus der Wirtschaft,
originally uploaded by Phil Strahl.

I went up to witness the last minutes of Alex McDowell’s lecture on the Production Design for Watchmen that was followed by John ‘DJ’ Desjardin’s presentation “The VFX of Watchmen”. Those guys are crazy!
I kept fighting against dozing away and luckily I won mostly because “Making of Dr. Manhattan” by Pete Travers from Sony Imageworks was very intriguing: From start to finish it took about nine months to develop the character visually as well as technically while keeping the VFX footprint on set as low as possible.

“If you have developed the best tracking system in the world but it hinders the actors you end up making perfect tracks of bad performances. Which are totally useless.” Pete said.

The actor playing Dr. Manhattan was wearing a suit covered in blue LEDs and tracking patterns and so he became not only source for video motion capturing but was also a very prominent practical light source on set. The tracking was done by triangulating images from the film camera and two Sony HD video cameras that captured the action from two more angles. The biggest problem in this approach was to sync the shutter phases of both the video cameras and the film camera to make tracking possible.

Another problem was the hue falloff of the LEDs: Close they where cyan, further away their light color became indigo. Hence the actor in the suit needed to be painted out on a frame per frame basis it became a huge amount of paint-work because he was a light source and every frame had to be painted separately instead of having a clean plate. So talk about painting hell.

While most of the people went directly to the Metropol theater to see Watchmen I really needed a break from all the highlevel-VFX and stayed for the “Animation Show of Shows”, introduced by Ron Diamond of AWN. I can get Watchmen on bluray at any store but might won’t be able to see some of this great animated films again. So here’s the list. I hope I didn’t mess up with the French titles.

• Keith Reynolds Can’t Make It Tonight, a witty stickman Flash animation that shares a lot with xkcd both visually and narratively.
• La maison en petits cubes, a hand drawn animation that tells the story of an old man rediscovering his past. A Japanese animation that looks totally French.
• KUDAN, a very abstract CGI animation about the relationship of a father to his child. Japanese. Weird. Breathtaking.
• La Queue de la Souris, a short minimalist tale of a mouse trapped by a lion. French. Witty.
• I slept with cookie monster, an analog animation drawn with pastels that tells the story of the animator that was abused by her lover and how she dealt with it.
• Franz Kafka: Ein Landarzt, probably one of the weirdest animations I’ve seen lately. Truly, the Kafka-esque spirit was captured very well in this short film.
• Glago’s Guest, Disney’s computer animated short of Russian guard Glago watching over endless Siberian snowfields.
• Hot Seat, The Office meets children’s cartoons. Funny yet true.
• Presto, a Pixar short I won’t get into because all of you know it already.
• Skhizein, my favorite today. The story of a man who is always 91cm besides himself.
• KJFG No.5, a very short animation where you’ll go “wtf?” at first. It is about a band jamming together that gets disturbed by a hunter. Great ending!

That was my day.

What I have learned today

• That foamcore models still are a big part of previz as are low-res interactive environments that every department can access.
• That it probably was a bad idea to jokingly refer to my new styling as “racoony” once — the word spread.
• That on very documentary films it is necessary to make sun and moon studies.
• That Sony’s Katana is da shit!
• That at Sony they deliberately decided against a house style and that they “challenge every assumption”.
• That Germans only realize how German they are if they work abroad.
• That you should start with 3d as early as possible in your previz for any shots that are not static.
• That a good way to ensure consistency in applying tracking-dots on an actors face is to make a plastic mask from his face, drill holes accordingly into it and then have him put on the mask: Make the dots through the holes and you’re done!
• That your VFX tricks on set should do anything but hinder the performance.
• That the scanning of skin textures should be done when the skin is anything but perfect or else you get the typical too-perfect-to-be-true CG-look.
• That eye moisture helps a great deal in the believability of a CG character.
• That instead of simulating rimlights in your shader (I consider that a no-no anyway!) you need to take the extra mile of adding peach-fuzz to your digital character. It renders longer, but looks much more convincing.
• That I really have to get my sleep cycle straight before attending the fmx.

What surprised me today

• That Oliver Stone looks like a chubby Albert Speer. Creepy!
• That my geekiness in terms of comic books is way below what’s common in the industry.
• That Zach Snyder draws really good.
• That I can sleep rather well on the floor of my car.

Room 212,
originally uploaded by Phil Strahl.

Currently I am in my bed at the Hotel Hottmann in southern Stuttgart. This year I got the room above the one I was in last year, unfortunately it is a bit smaller, still comfy. Most important to me is the radiator next to my bed because I freeze easily and the room was really cold. Even colder than the hallway. So I turned it all the way up and spend my time waiting for the WLAN to transmit data-packets, one at a time at about 25 baud. Every 5 min I have to attempt reconnecting for another ten to thirty minutes, so it’s quite tedious. But not as tedious as driving around for 20 minutes looking for a space to park the car — Stuttgart’s jammed!

In all the excitement of today’s journey I totally forgot to prepare me some food and all I ate today were gifts: A piece of sausage, a sip of apple juice and 24 thin Hello Kitty wafers. Now I count the hours until they serve breakfast around here – 6.5 more to go – d’oh!.

Continuing last year’s list of stuff I forgot at home there’s also one for today.

• shaver
• soap

What I have learned today:

• That sunsets still touch me.
• That 100 calories won’t make your day.

What surprised me today:

• How cold the Hotel Hottmann’s rooms really are.
• That a woman really said: “Hey, this cloud looks like the Enterprise!”
• How easily annoyed I get, when I can’t check my mail.