FMX ’10, Day One

I awoke after a terrible night of too little sleep (thank you, insane entertainment-industry sleep-cycle!) and was greeted suspiciously by Mrs. Zheng, the hotel manager, on my way to the hotel’s breakfast premises where the ongoing conversations ebbed as I entered. Too much eyeliner, I thought. But I had other things on my mind. In fact, I was so excited that I ran a red light on my way to the conference.

I was eager to first see The Foundry‘s presentations on Mari, their programming approach and a tech demonstration of the recently acquired Katana, I was so excited about last year in Sony’s presentation, but had no exact clue what it really was.

Paint that dinosaur!

Once arrived I got me a seat pretty close up front and was ready for their presentations to begin. Jack Greasley, who worked at Weta Digital on King Kong and Avatar and Zoe Lord, Senior Texture Artist on Avatar, presented Foundry’s upcoming texturing tool Mari.

Weta needed a decent and perfomant tool because handling the characters of King Kong for Peter Jackson’s movie was a challenge itself. The monkey was huge and complicated and worked more like a complete set than just a character. And after learning about the tedious process, everyone at Weta knew, that Avatar would become just as exhausting and complicated like Kong — only multiplied hundreds of times.

The first version of Mari was took 16 months to develop and once it was running, it was constantly in production and used by the artists in production until up the current version. Although Mari is now internally in version five, for its public release Mari will launch as v1.0

Essentially Mari is intended as middleware between sculpting and animation, and effectively completely replacing Photoshop for a texture artist. Since this tool has been developed at Weta Digital for a couple of years now, the workflow in Mari is rather straight forward: You import hundreds of still images that can be manipulated inside the software and applied directly to the model you’re working on. You can perform 2D operations to your references such as cropping, color correcting but also warping or pinning. The performance is outstanding, the software handles well over a 100 2k-maps in realtime on a multi-million poly model, and here’s the best part, Mari can read and play back .obj-sequences of an animation so an artist can correct ugly stretching errors in the texture on the fly. this performance allows the artist to load and work on whole sets with moving objects and to create seamless textures across objects more easily. Did I mention that you can, of course, animate your textures when needed?
Mari is also capable of rendering occlusion-passes which can not just be multiplied to the color-maps with blending modes like in Photoshop, you can also use them as masks to paint dirtmaps.

“We realized that with other tools the artists were much more data wranglers than working creatively, setting up map-channels, importing files and shading networks and so on.” Mari offers the artist various tools and guides to work effectively and to avoid mistakes such as a protection system for edges or the possibility to use channels as masks.

I wanted to know about the exporting capabilities of the tool, like whether it was possible to export your shading network from Mari straight into Maya’s hypershade. “Unfortunately not, you will need to export your maps as separate TIFs, but the SDK is very open so you can have scripts in your pipeline that do that for you.”

Also displacement maps are currently only previewed in Mari as bump, yet The Foundry works closely with nVidia and ATI to add certain features.

And you’ll need at least a gig of free video memory to allow Mari to unfurl its glory.

6.1

Simon Robinson continued with what’s new and cool with Nuke 6.1, such as the refined camera tracker or “stuff that’s not really rocket-science but still needed to be done like moving a camera while looking though it”; “GeoSelect” to select points of a point cloud; the modeler node that less you create geometry between points of a point cloud, which works a bit like in Boujou, but with the advantage of refining the corner over time yourself and have Nuke fix the track cleverly. This seems to be the way to go in terms of scene salvage in steroscopic productions, where you project your paintwork on such geometry in your scene. The WriteGeo node now also lets you export .fbx files, not only load them.

Another thing totally interesting is baseline camera correction, which I didn’t really grasp in the short presentation, but it appeared to let you stitch panoramas together and create 3d-vistas out of them, kinda like photogrammetry.

And in addition to Keylite and Primatte, Nuke will also support the Ultimatte keyer, so Nuke will have all three industry standard keyers available.

Simon also provided an outlook of further Nuke releases, mainly improvements in stereo roto & paint. When asked in what Nuke version this or that will be available he would just answer with a shrewd smile: “In Nuke 6.n, with n bigger than one.”

The Foundry is planning to release Nuke 6.2 this summer and expects another release at the end of this year, most likely featuring the first helping of adopted technology from Katana.

Making it faster!

This was the title of the following presentation by Bruno Nicoletti, CTO and co-fonder of The Foundry. He stressed that there has always been the problem of any software nothing is ever fast enough and before continuing made a disclaimer: “It’s going to be very technical” before he continues to get to heart of the problem’s possible solution, the GPU. “Many people are obsessed about the GPU and we are not taking quite the advantages of it as we could.”

GPUs are really good at image processing and programming them is easy. But getting peak speed is hard for programming GPUs is different, and there are lots of new and interesting technologies like CUDA or OpenCL. Simple tasks such as blur and color correction pose no problem, but graining or motion estimation is quite hard to do, even with CUDA it still is complicated.

The Foundry codes everything in C++ and less OpenGL. If any OpenGL code is needed (like for Mari) it is handcrafted and different for every application because “We want the best performance on all devices” Bruno boasts.

So what are the problems? The first problem is that GPU code isn’t as fat as it could; then everything has to work exactly the same on all devices. For basic stuff that’s easy, but producing the exact same results for two completely different code-streams (C++ and CUDA) is hard and almost impossible to manage. Manually optimizing code is a complex specialist task and works differently for every device, makes the code much less legible, is tedious and slows down delivery profoundly. Or what happens when new hardware gets out? Then every bit of code needs to be adjusted to deliver the same results, manually again, which makes the product prone to bugs and, again, the process is painful and expensive.

So the task was to produce code only once, that needed to be clear, fast, legible and easy to port. The Foundry came up with what they call “Blink“: A programmer writes abstract C++ kernels to process images. The code for a certain device can be translated from the original code automatically for various devices such as CPU or GPU, taking the advantages of either. Bruno was showing a demo of this with the Kronos re-timer, running on a GPU via CUDA. “But this is only the start”, Foundry want to continue with SSE and OpenCL and yet more as well, clever processing graphs and run-time code generation.

The upcoming versions of Nuke will also incorporate these changes over time. Since Nuke needs scanlines whereas the GPUs work with tiles these changes will be additions, because “Nuke will always be processing processing scanlines” Bruno assured the audience, “but we will be implementing this into some realtime nodes, so sections of Nuke may be redesigned to work with tiles”.

As the demo showed, the GPU is surprisingly faster than the CPU with Kronos, although it is highly dependent on the chip itself. Yet it is possible to mix GPU and CPU calls for an overall higher performance. Since motion estimation already works this way, it can be expected that this will show up sooner or later in the Furnace and Ocula plug-in sets. “And it is even possible to run it via a host application”, so I guess that’s good news for all of you After Effects and/or Final Cut users.

I Want A Pony!

Andy Lomas continued the Foundry session talking about Katana. Since I first heard some squishy descriptions a year ago I wondered what Katana actually is. Now there’s a press release and a FAQ on it on The Foundry’s website.

As we all know, Katana was developed in-house at Sony Pictures Imageworks for the past years as heavy-duty lighting and compositing package. But that was about it. If I had seen this article on fxguide, I would have already known. I strongly suggest you should read up on it, because I won’t repeat here in my clumsy words what has already been put to hypertext already.

However, I will quickly sum it up to the lazy among you: Katana is Sony Pictures Imageworks proprietary node-based tool for lighting, compositing and distributing render-jobs. It is asset-based where you can load your assets from your favorite 3d package and manipulate every little detail, when you want to also on a per shot basis. Katana sits on top of the assets, so it’s totally non-destructive and rule-based when through it overrides are performed.

This allows the artists to start lighting a shot, when the assets are still in production, for example, and to develop a look early on. Thanks to clever versioning it is easily possible to produce a number of suggestions and to get back to the right one easily.

Sony has developed its own format for Katana, one which handles assets as assembled components in a hierarchical structure, e.g. a city is composed of various blocks, which consist of various buildings and each building has a roof and so on. Katana won’t lode the full scene graph until the artist exposes it. Only what gets expanded in the hierarchy will appear visible in the viewer.

An interesting thing is that Katana is, according to Andy’s slides, render-agnostic which means you can set up your render passes in Katana and decide there for a renderer such as Arnold or Renderman, you can even easily plug your own renderer into Katana: When your renderer understands the concept of a shader, Katana can work with it. A huge advantage of this is, that you have a consistent interface from lighting to finish, and work in one environment all the time.

In Katana you can choose what attributes you want to expose and/or modify to the renderer. The full assets with high-res-geometry and textures only get loaded via the renderer, so you only deal with what’s important and don’t get lost by a cluttered scene: What Katana can defer, it will defer. Also in Katana you can trace why something is rendered the way it is, like where an asset gets its specular map from and so on. For example, a Ds tag means that the value is the default from a shader, whereas Ls indicated that the value was set locally from a node within Katana, so an artist can always “debug” shading and lighting issues.

If you want to get your hands dirty with Katana yourself, you can: There’s an API for your own company’s C++ plug-ins and, of course, Python support. The scripting language in Katana itself is CEL which can be used to select certain nodes, e.g. to apply a certain material to all geometry nodes that have a certain name match, certain tags or attribute matches but also by a collections. This allows to override parts that do not exist at the time but will get modified once they are ready. There are a lot of further options to use CEL to override multiple object, like only turning on the specularity of a whole scene — bang! — you just made a specularity pass.

Now let’s have a closer look on the user interface and the workflow with Katana. Andy showed the audience what instantly sold it to me, namely the “I Want A Pony” menu option, which creates a pony-shaped node in the graph view. “This is actually one of the horses of Beowulf, so it’s technically not a pony. But we use it as a primitive here in Katana. And it might even succeed the Utah Teapot” he added jokingly. The further now one drills down on the asset, the more parameters get passed on to Katana until you arrive at the vertex-level. In theory you could make whole 3d-animations and models in Katana.

Compositions look like trees with roots: On the top you load (and/or) group your assets and materials (like branches and leaves), assign nodes that perform certain changes and in the end you have multiple renderers and render-passes (like roots). And the interface is, of course, very customizable and panels can be docked or torn off.

Materials can be stacked together into one stack which has only on input and output connection to the outside to keep the complexity low where it is not needed. Everything can be inherited to child nodes, even transformations, if you want to. So for example making a wet material can be just done by creating a child node that inherits everything from its parent but diffuse and specular values.

The gaffer-node is a “one-stop-shop-node”, as Andy put it, to execute a macro and to create lights with the most common attributes already exposed.

Katana composition can be references or, by using the KatanaSdtBake-node, be baked for other artists to use, have live groups in a macro or share real Katana scene graphs (i.e. the compositing-scripts) via a library.

In the end, Andy announced that Katana won’t probably ever ship as a single product by The Foundry. If it ever will, then probably with its 2D capabilities stripped out of it which will in turn be used in a new Nuke version. Katana features in Nuke will probably show up still in 2010. There is potential in Katana as a re-lighting engine, “but that’s quite far down the line at the moment” Andy concluded.

35 Years of Slapping your own back

Lynwen Brennan’s lecture on ILM‘s past 35 years was as mundane as it was boring, but it was no surprise hearing that from the President and General Manager at Industrial Light & Magic. At least her presentation had many pictures in it (in the 70′s, every guy had an impressive beard), tables and schematics of why ILM is the most successful company there is; that ILM invented digital editing; that John Knoll and his brother came up with Photoshop; that the first shot featuring digital compositing (and not a mere optical one) was in The Abyss — so it was a nice blend between appearing all corporate and fun facts. Lynwen was reading her hour-long presentation that left no questions open, partly because of the fact that it contained nothing that you wouldn’t find on Wikipedia. The primary message was that ILM tries to raise the bar, keeps the costs in check and wants to define new standards.

What followed was a panel discussion about the global production at ILM, since ILM has also a studio in Singapore (that’s where the Clone Wars series gets produced, including the Nintendo DS game, see last year’s fmx coverage), and works closely with other post-houses such as Pixomondo.

My lack of journalistic skill now really shows because I didn’t get the name of the moderator — my apologies!

The panelists were Dennis Cooper, Lucasfilm’s Director of Global External Production; Gretchen Libby, executive in charge of external production at ILM; Mohen Leo, ILM’s Singapore Studio Supervisor and Thilo Kuttner, CEO of Pixomondo whose biggest contribution to the discussion was the fact that he held his microphone like an umbrella all the time and appeared overall subsequently rather faint (see photo below).

The first question asked to the panelists was a basic one: Why global? In ILM’s opinion it’s a good way to keep the cost down and to maintain quality by having many options. For Pixomondo (who have offices in Babelsberg, Frankfurt, Munich, Stuttgart, London, Shanghai and most recently Los Angeles¹) being global is not only to leverage the global talent but also a way to keep everybody busy.

Since communication is one of the strongest factors in collaboration it neither ends for Pixomondo nor for ILM at simple email exchange. Thilo strongly proposed also understanding the different mindsets and cultures instead of forcing your own ways upon your collaborators. To ILM it is important that every bit has to be as clear as possible, “It’s not only about being clear in what you say, it’s also considering what the other one hears”. Mohen further stated that for the artists in Singapore it is important to have direct access to their respective supervisors in the US. “The better you know a person, the better you know what he or she means when saying something. The simple act of having lunch together can help a great deal in that respect” Gretchen Libby added.
The diverging mentalities really do put international communication to a test, in Singapore, for example, direct yes or no answers are avoided so different strategies needed to be sought. “It’s important to know that when you ask ‘Can you do the shots till Monday’ and you get ‘We do everything we can to have them on Monday’ it actually means ‘No’.”
Pixomondo even introduced a command-list to get the most urgent and most important communication dead-right, independently of the culture. “At ILM we know we are all in a very visual field, so we draw a lot of pictures.”

To ensure a continuity in production ILM is a big proponent of creative leadership and development of long-term relationships. “You don’t produce where it’s cheapest, you go where you get the best ratio between quality and price.” In terms of production pipelines Pixomondo orient themselves on ILM, who are really fond of their proprietary stuff, yet they insist that they are open to change.

The process of going global (or not) starts very early on for ILM, when they look through the script. Based on this script they assign different tasks to different studios all over the world, studios with strengths in certain areas, although “we look for synergies as well.” And ILM goes “where we can find capability and punctuality”, both equally important in any big production.

“But is working together with other studios not like training your competitors?” the moderator asked with a wink. But the folks of ILM stayed relaxed. “Talent and competition are both everywhere [..] we rather look for people who want to work for us”, to them everything is about building long-term relationships with their partners. “Also our clients want and need to know where their material is being worked on”. For Pixomondo working with ILM was also a long-term strategy as Thilo added, “not least because it’s an honor to work with ILM.”

In the VFX business (especially with big players such as ILM) security is a hot topic. The process of evaluating the security standards of a potential partner begins very early on, “basically with the first phone call”. They need to fill out a questionnaire on who has access to their premises, to their data, what kind of passwords they use and so on. “And site visits. Lots and lots.” The material they get to work on is watermarked, data transfer is password protected and so on.

So what about the future? Thilo concluded that the business will continue to go more global (big surprise), and also ILM only wanted a “continuation of where we are now”, perhaps with a 24-hour feedback cycle; an attempt to work with the time-zones instead of against them.

The usual colors

The visitors in the König-Karl-Halle had thinned out a bit for the upcoming lecture by Technicolor’s Tim Sarnoff on “Working Across the Globe”. Maybe I was a bit tired by the previous presentations or the lecture really lacked much structure and in the end there were even less people in the audience. In the end it was just a company presentation of Technicolor with the occasional buzzwords thrown in, mixed with some commonplace information such as “You remove a certain degree of risk when going global; One has an unlimited talent-base when being global; Being a global company needs work and consciousness” and from the Technicolor promo video I noted down “It’s a business of passion enabled by technology”. I was glad when it was over.

On the plus side I swapped business cards with a German screen writer who had her degree from the prestigious Filmakademie Baden-Württemberg and for lack of job opportunities in Germany (“I hate German drama!”) she resigned to writing a novel for the moment. Networking on the fmx was as easy as always.

Down the rabbit hole

The last lecture of Tuesday was by Ken Ralson and David Schaub from Sony Pictures Imageworks (SPI) and all about Tim Burton’s Alice in Wonderland in which they lined out some of the production process. “First thing we learned was that Tim [Burton] hates storyboards. So we started with gathering a load of reference material with our starting point being the 1922 Alice in Wonderland movie. In theory they wanted to establish a style and then shoot which in practice didn’t really work out, in the end changes needed to be made until the last minute.

For the pre-production they found an artist at the portfolio section of the CG society² they liked and gave him a little piece as a test. They liked the result and hired the guy from Germany. “I bet the CG society will get a lot of new members as soon as this lecture is over” I thought to myself.
The concept artists painted a lot of designs and suggestions until Tim Burton would eventually say “This feels right” and the artist could ponder about what it was that felt right to Tim and develop it.

What SPI did in the beginning was also a test of enlarging Johnny Depp’s eyes in a shot from Burton’s great film Ed Wood. The result was hilarious and “when we showed it to Tim and Jonny they both cracked up so we knew it would work. [...] and after a while working with it, Johnny’s real eyes seem always a bit too small.”

The principal green-screen shoot was incredibly tight so everything needed to be planned and considered in advance. “It wasn’t an easy task considering there was so much that needed to be CG, like Crispin Glover whose whole body needed to be replaced, or Alice changing between three different sizes throughout the film”. All shots with extras or supporting characters were shot separately.

Since the set was, apart from the occasional set pieces, totally green, the team of SPI even installed a pre-vis system on set for Tim Burton to watch. “He looked at it, then grabbed the monitor and turned it away. Tim preferred looking at the raw greenscreen play-out³ but that’s okay.”.

The raw footage from the greenscreen stage they presented looked, truth be told, incredibly silly: Mia Wasikowska running on a green treadmill for her life or riding on a green upper half of the Bandersnatch that was shaken by some grips to a click-track; Johnny Depp standing heroically in a crazy costume with weird makeup; Anne Hathaway in an extravagant costume riding on a green vaulting horse carried by three guys acting all horsey in green; there was Matt Lucas plus stand-in in a green pear-shaped costume with tracking markers all over as the Tweedles; Crispin Glover in green shoulder-pads every quarterback would kill for walking on stilts, trying to act all normal while two guys in green were always walking next to him, ever alert to catch Crispin in case he would trip. Yes, it looks absolutely retarded. If it wasn’t a multi-million dollar production you would just laugh, then weep and then facepalm.

In order to realize the big head of the Red Queen, the shots with Helena Bonham Carter where were shot in 4k resolution and everything but her head scaled down 50%. In the scene introducing the Red Queen she wipes a drop of jam from one of the frogs’ faces and sticks the finger in her mouth. To realize this interaction her hand needed to be painted out of the plate with her head, a terribly challenging piece of paintwork. And thanks to my contributions to Ninja Assassin I know what I am talking about.
Another demanding paintwork was Stayne’s hair: All the concept paintings of him showed a knight with big shoulder pads, so for the greenscreen shoot Crispin Glover was put in a green costume with said shoulder pads, yet in the end Tim Burton made up his mind and went for an armor without shoulder pads. Needless to say that the missing hair needed to be painted back into the shots.

Another thing about Stayne was his height. As stated above, Crispin was walking on stilts on the set but capturing his movements on set resulted in an awkward animation and looked much like it looked on set: Like a guy walking on stilts. So the animators resorted to good old manual keyframe animation for Stayne while keeping the essence of the movement on set for him. This reminded me of a quote from Pete Travers on The Making of Dr. Manhattan at last year’s fmx: “If you have developed the best tracking system in the world but it hinders the actors you end up making perfect tracks of bad performances. Which are totally useless”.

The appearance of the Cheshire Cat was easier in contrast. The animators started out with a very cat-like animation but Tim dialed them down until the cat was almost not moving at all. And to achieve the terribly wide grin, the cat’s jaws needed to transform as well.

Since the feature was in stereo ⁴ (and what isn’t nowadays anymore?) but shot with only one camera, the stereo-conversion of the live-action footage was done in post, by rotoscoping and/or projecting the live-action cards on 3d-geometry within the scene.

After this refreshingly interesting and engaging lecture it was also this year the time of Shelly’s Eye Candy Show.

Sweets for your eyes

Also this year Shelly Page from Dreamworks Animation assembled 50 minutes of animations she considered highly worth watching. Some would also get screen during the International Festival of Animated Film I also attended after the fmx was over, but more on that later.

The screened films this year were:

• Mobile by Verena Fels, a cute little animation about stuffed animals hanging on a mobile with a funny pacing and rewarding pay-off in the end.
• Stylo, animation by Passion Pictures, the Gorillaz music clip featuring an amusingly smug Bruce Willis and artful integration of cartoon characters in live-action footage. “Stylo is directed by Jamie Hewlett and produced by Cara Speller for Zombie Flesh Eaters, with live action through HSI Productions in Los Angeles, and animation by Passion Pictures in London”⁵
• Log Jam Series by Alexey Alexeev, the first clip formerly known as KJFG No5 ⁶. This hilarious little animation was also in Shelly’s reel last year but a producer urged Alexey to make make more, for a series. So he continued with some more, all equally pointless yet entertaining. Be sure to watch them all, they are called “The Rain”, “The Snake”, “The Moon” and “The Log”.
• Anchored by Lindsey Olivares, her senior thesis film made at Ringling College of Art and Design after Romans 15:13. I enjoyed the style of flowing watercolors much, although the topic wasn’t so much my thing.
• Inka Bola by students at Gobelins, an entertaining piece about a spoiled toddler and his guard. The animation is superb (but frankly I don’t expect anything else from Gobelins) and the whole piece has the speed and style of a Disney or Dreamworks short. Very enjoyable.
• Evian Babies animation by MPC, a weird trip to the outskirts of the Uncanny Valley: “Michael Gracey has directed Evian’s latest commercial Skating Babies a multi-national campaign bringing together choreographed roller-skating babies and the re-mixed street sound of The Sugar Hill Gang’s Rapper’s Delight. Created by the agency BETC Euro RSCG, the spot was produced by Fabrice Brovelli, Head of TV at BETC and Jaques Etienne Stein at Partizan. MPC created fully CG baby bodies and carried out extensive live action head replacement and compositing as well as large scale digital matte paintings to extend the park environment for the TV and online campaigns.”⁷
• 5alive Dodo animation by Passion Pictures, a TV commercial with funny character animation of a dodo dancing to I’m Alive by Don Fardon.
• LowLow Cheese animation by MPC, another TV commercial with a photo-real mouse avoiding a crapload of mouse traps. Yes, we’ve come a long way from Stuart Little.
• RockBand Beatles intro by Passion Pictures, also this year the RockBand series has yet another stunning intro to worship. In just two and a half minutes the intro tells the story of The Beatles’ success in something I just call “masterfully art-directed pictures” (and sound!) and is highly enjoyable to watch. I even found a short behind-the-scenes talk about it on YouTube!
• The Little Boy and the Beast by Studio Soi. Oh, I just love that one, it’s about a boy with a depressed mother and how he deals with the situation. This is not only a tough topic for children, it is also skillfully executed and designed, from the plot to the final playout. It also ran on the children’s section at the International Festival of Animated Film and the kids loved it. To me it was the best short on Shelly’s reel.
• Lost and Found by Studio AKA, a 25-minute short of a boy confronted with the sudden friendship of a penguin. It was nicely done, though there was a lot that bothered me such as the narrator off-screen that not even commented what was going on but only repeated what the pictures showed; the sequence out on the rough sea which lasted waaay too long and the water itself that was too photo-realistic to fit convincingly into the style. It was nice, yes, but about ten minutes too long and nothing spectacular — sorry.

Phew! What a day! You see now why it took me so long to get this from my head to my blog, there was just so much knowledge to chew and digest. I hope you stay tuned for the upcoming reports of the following days. Hopefully not as lengthy, though.

I awoke after a terrible night of too little sleep (thank you, insane entertainment-industry sleep-cycle!) and was greeted suspiciously by Mrs. Zheng, the hotel manager, on my way to the...

Leave A Reply

Name

Email (will not be published)

Website

Comment

Notify me of follow-up comments by email.

Notify me of new posts by email.