Okay, I lied, but it should be at least funny.

A bit.

So this is what happened:

So after my last post I spent some more nice days in Berlin, got treated unfairly by Air Berlin and spent eight more hours than expected at Tegel Airport but met a nice couple from Salzburg that just got engaged in Berlin. In fact, I was just fifty meters away when Stefan proposed to Julia in the Mauerpark. I was busy looking for presents for me, my friends and myself. So one woman got a ring and I got some games for my Atari 2600 — everybody was happy.

Back in Salzburg I had a shower and the next day drove to my lovely and talented digital-artist-girlfriend in Tyrol where she stayed with her parents for a couple of weeks. Whereas Berlin was cool and rainy, Tyrol greeted me with hotness and sunshine. And Conny and I walked and even hiked quite a bit. Her family was super-friendly and it was a nice vacation from my vacation.

Not so long after I had my birthday, my very good friend Jot, the game designer finally moved out of the campus with his girlfriend. Not long after I witnessed with Conny the last few of her colleagues’ Bachelor exam. It was that day when Joey, head of the animation department at the Salzburg University of Applied Sciences, asked me to manage the students starting this year in the animation master course, since “you are in the project-management master class anyway.” I agreed and so the semester started for us a bit early but creatively.

Hey, and I bought my dream-woman a shiny ring with a shiny stone and asked her to move in together when we’re through with our Master’s degrees. And — yaaaay! — she agreed!

And before we knew we were sitting amidst about a hundred people in the biggest lecture hall for the introductory presentation of the Master’s curriculum. I am not afraid to say: Finally! As a student at the FH Salzburg I’m as happy as a lark and started out way too keen to do well, taking notes, reading up on topics and managing the animators. Hell, even the night-shifts at the Red Bull Media House in conjunction with very early courses the following days don’t scare me, they just exhaust me a little. But hey, I can sleep when I’m dead, right?! Which may happen sooner rather than later if I keep up this lifestyle.

Gadgetwise

Yeeees, this is the part where I let my electronic bling shine: As said before, Berlin was a retro-computing kick-starter and I returned with an Atari 2600 Jr. plus some cartridges in my hand luggage. The Atari still works, as Jot and I found out in one heavily documented gaming evening. It was a load of fun, despite the very noisy TV-picture which made it hard to make out what was noise and what was a bullet.

Then, of course, Propellerhead put out a new release of Reason wich now integrates Record plus at last a studio-grade mixer and 64-bit support. Sugoi! But the most awaited gadget arrived a few days earlier, something I had waited for almost two years since I first saw it in a YouTube video. It finally shipped to my address from Hong Kong and despite the odds (UPS’ inability to find my address for four years, UPS’ finally calling me with the instructions where I should go to retrieve my package, plus UPS’ invoice charging me additional fees to the already royal amount that customs already took from me) I had it in my hands: The Misa Digital Instrumets Kitara, a fully digital guitar with built-in synth behind a multi-touch panel running Linux. Yes, the awesomeness was oozing from every inch of its black and shiny polymer body. As soon as I had some time I plugged it in and realized that I had neither a clue nor innate talent in playing this instrument. But DANG! I’m looking gooooood with it!

Recently I grew increasingly annoyed with HTC’s Android distribution on my mobile phone, the Desire Z or G2: Buggy audio profiles, promoted applications like Amazon MP3 you couldn’t delete and an insatiable hunger for memory. Once I was fed up enough I posted my misery on Google+, asking for advice on how to flash the device with a custom ROM. I got the answer, got it up and running (I might lay out the details of it some time since it was way harder than anticipated) after four hours and was so happy that I also flashed the firmware of my Canon 5D Mk. II with Magic Lantern. My recommendations for both ROMs.

Anything else?

Well… no. Not really. But at least I got some little creative stuff done, like a SNES-music track for a boss fight in a game my dear friend Jot, the Game Designer is working on. I thought my piece is sub-standard, mediocre crap but he likes it. Poor fella. Lost his mind obviously.

Another thing I always wanted to try was programming apps for my Android phone. Installing the development environment was so tedious and complicated that I really congratulated myself when I got it to work, because I thought that it was the hardest part of my career as a successful programmer. Oh, how wrong I was: After two or three days of heavy research I was at least able to code an “app” that force-closed when you tapped any of its sparse user-interface widgets. A few days later I realized that it might be a good idea to properly learn Java before attempting to code The Best App In The World. I got me a book (thanks Conny for advising me in the bookstore!), I got my head wrapped around that whole object-oriented crap (hey, at least no pointers, headers and memory management like with C++) and I even managed to code an arrow that one could control like a car from a top-down perspective. That’s where I stopped for now.

But I got back to my Android phone, but this time more on a creative side. When you got root-access of your phone and a custom ROM, there’s not much you can’t do — gawd, I love Android for it’s openness…

So?

So that’s it in a nutshell. If there’s any possibility that you might have read it all and not just clicked the photos in half-hearted anticipation of seeing something shareable on Facebook, I want to congratulate you and apologize for taking away all those precious minutes from your life-clock. More to come soon. Maybe. If I find the time.

Market

The closer I got to the Mauerpark, the more people filled the streets and once inside the premises of the market, everything was tightly packed – as ever. I had a list of people I want to present gifts from Berlin and the flea market is always a good source of unique presents one won’t get anywhere else. And eventually I find something for myself, I thought and mingled with the crowd. But before buying something, only to find something even better a few steps away, I decided to walk and spot the entire marked without making a purchase whatsoever.

For over two hours I looked at thousands of goods, always in search of The Perfect Gift in the back of my head — and maybe some retro electronics for myself as well. Unfortunately the market has quite been picked clean of good old game cartridges and consoles. In total I spotted just a single, lone Sega Genesis in poor shape, an okay Atari 2600 next to a very dirty SNES and the occasional Nintendo DS. The handful N64 games scattered around various stand were either the usual suspects (Mario 64, Zelda) plus some obscure and most likely crappy sports games. The GameBoy cartridges featured the same “diversity” of titles and mostly were in terrible shape (ripped, bleached and torn cover-stickers!).

The only good thing really seemed to be the Atari and instead of considering what presents to get for whom, I caught me just obsessing about that console from 1976 and to keep me from worrying too much I bought it for 20 bucks which was less than I had expected since it came with some cables and a Sega controller (that is compatible with the Atari, of course). I was happy.

Now I could look for some presents. And I found one: Another present for me: A 1977′s Pong clone, titled the Unimex Mark III in very good shape (original packaging) and a real bargain for 7 €! I continued through the stands and even stocked up on post cards until I finally, finally found some presents from other people. I felt so good about my selflessness that I had to get me a reward. That one vendor sold hand-made watches and adjusted them to his customers’ wrists flabbergasted me profoundly and in my excitement I got me a classy custom watch for classy 29 €.

Before leaving the market this time I wanted to get something healthy to drink so I settled for a bottle of freshly pressed pomegranate juice, the weirdest thing I’ve ever drank: It was very sweet, very bitter at the same time, very sour as well and overall very intense. And it tinted my ‘stache probably very red.

I suspected the café Kauf Dich Glücklich to be helplessly overrun and besieged by hungry tourists, like the Glücklich am Park I spotted later. That’s why I headed straight for the Café Napoljonska instead, got a seat and a waffle but was disappointed by the change the whole place had gone through since my last visit last year: It was now cleaner, bigger, brighter and had gotten renovated. What once had the charm of a cozy little café was now a colder, less individualist place that didn’t have that certain Berlin-vibe to me. At least waffles and café were superb. When I felt exhaustion creeping in on me, I left and walking made me feel better.

So I continued up the Kastanienallee to the Lichtblick movie theater to fetch their program. On my way there I noticed another change: The once dark red (i.e. communist) Morgenrot Bar where you paid for a breakfast “as much as you saw fit”, is now also a bit cleaner and has streamlined their name as well: Morning Glory. I hate to say it, but the more renovated and improved Prenzlauer Berg gets, the less and less it is attractive to me as a place to roam and explore.

Dennis

At Rosenthaler Platz I boarded a train back and when I arrived at Esther’s she was about to leave for the Konni Café at Kottbusser Tor meet with another former fellow student, Sandra, who was also working and living Berlin now. Esther took the bike and I wanted to take the train. I missed it and decided to walk instead. At Lausitzer Platz I passed a guy in his early twenties with a slight speech impairment and obvious autistic disorder who asked me to take a photo of the church.
“I can’t man, I’m late!” I said and looked back at him. It obviously encouraged him to follow me and so we walked along down the Skalitzer Strasse. While I was thinking how I could get rid of my unexpected companion, he wanted to “touch pants” and was quite assertive. Obviously he liked my black denim jeans.
“What’s your name?” I asked him,
“Dennis. I wanna touch your pants!” he kept going and I gave in at a red light. He wanted me to stand on one leg and he laid his hand on my upper thigh, watching closely.
“Touch pants!” he said once more and I noticed a dictaphone in his hand. When the lights turned green again, finally, I asked about the gadget.
“I like to record train sounds” he said. It appeared to me that he just keeps riding trains the whole day, recording his experiences. A bit like me with my blog. On the next red light he shouted
“Touch pants! Touch pants!” I sighed and put up one leg. Dennis was intently touching my pants and bystanders looked shyly.

We continued to the Kottbusser Tor and Dennis grew more and more impatient. I told him on the walk before that I was meeting friends at the café there and he had a hard time catching on, so I kept repeating it over and over again.
“Can we sit in café and touch pants?” he asked and grabbed my arm a little bit aggressively as we crossed the street.
“Dennis, please let my arm go!” He did but kept asking me to “sit down and touch pants.” To make him happy I sat on the stairs leading up to the café blocking all pedestrian traffic and he wanted me to take out my phone to he could better “touch pants”. Esther was watching us with a puzzled smile.

I introduced Esther to Dennis and the same moment Sandra arrived and it was a very confuse situation. I let Dennis touch him my pants one last time, although he kept shouting
“Longer! Longer!” but at one point the three of us cut him off and told him that he had to leave. Before he did he asked me whether I will be passing by Lausitzer Platz again.
“I don’t know” I replied, taking the decision that I won’t be passing there. Dennis was a friendly guy at heart but he was too complicated and stubborn for my taste. I wasn’t looking for a companion tagging along, I was looking for a few days of vacation. Dennis finally left to the Kottbusser Tor station and Sandra asked for how long I knew Dennis.
“About 15 minutes” I replied to her surprise.
“You guys looked like you got along nicely” she smirked.
“Touch pants!” Esther concluded with a broad smile. There was no point in trying to repel the formation of an in-joke at this point.

Sandra

Sandra just came back of a “date” (her quotation marks) and she wasn’t sure about her feelings and seemed more shy than had known her to be. Esther and I tried to help her sort out her emotions but we constantly drifted off into jokes. Paradoxically this seemed to work out for Sandra. And when a young friendly dog with a bandana looked at Esther, who instantly fell in love with it, the topic was more or less dropped. When Esther had to leave for her contact-gymnastics-session, I talked with Sandra for an hour longer and she invited me to tag along the next time she and her work colleagues from the animation manufacture she was employed. Back home I took the subway this time.

Now Esther is lying in her bed and listening to a relaxation video on YouTube, since she has troubles sleeping. Frankly, this is getting me sleepy as well… good night! If only the guys across the street could quit yapping.

I only somehow woke up after my caramel macchiato shot (that I would have preferred injected directly into my heart but the lame-ass barista refused to) and I found myself in the Großer Saal. Second row — How did I do that?

DAY 4 IS COMPLETE – W00T! (at least the text is)
Soon I will finish writing the other days’ reports, then add lots of pretty pictures and proof-read the whole grammar-abomination thoroughly.

The Studio

The STUDIO from New York were giving the first presentation, STUDIO as space. Interestingly you could instantly tell that both presenters, Mary Nittolo and Gary Giambalvo were from New York. And their names rhymed, they both had an Italian sure name, the same hair-length, and I think they even had the same glasses — charming! Mary and Gary started off by showing a digital mural from their website, that was a collaborative piece of some of STUDIO’s artists. The concept of STUDIO is creating a community of functioning teams, especially with artists and freelancers that join the environment for just a couple of weeks before they are off again. Mary was one of the few employers that really gave this a though, how such freelancers feel when they arrive as strangers in an already established community, “there’s no constant environment when you freelance.”

Mary had the idea to the STUDIO in Italy, when she was observing a painting in a small church: The painting was done by multiple artists that all built upon what was already there, different people came together produce a piece or art together. Twenty years later the STUDIO is operating that way, although “it has become a bit difficult in the current market. But we’re close to a new renaissance of art and science.”

STUDIO has 20 employed artists and also offers places for interns. “We want them to come with a project,” Mary added and showed “The Sparrow”, a project by one of STUDIO’s interns who had a very illustrative style and did some character development sheets, a colorful and well elaborated storyboard and a very elaborated animatic of it, complete with music and sound.

But Mary also wanted to engage in an exchange, asked the audience questions about their two cents about working freelance. How was it to work and live in New York? “The problem with New York is finding a place to live you can afford. But if you are persistent and really want it, there’s always a way to figure it out,” Mary encouraged the audience, mostly assembled of students and freelancers at it seemed, and one zombie running on caffeine¹ So why not just work from home as a freelancer then? “Working alone is isolating. It may be a bit tough to arrive in a new environment but exchange can and will happen. Working alone from home is the most alienating, because you will always have to find a person in the studio who can give you feedback, that the working experience becomes valuable to you. It should be you to initiate that contact, a lot has to come from you that way. So you should ask for it to prosper from it.”

Despite producing small projects of their own, the STUDIO started out as a storyboarding facility — and still is. “I have seen many storyboards of you guys here and the storyboards we do are much more specific and detailed.” Indeed, the stuff Mary showed was really well done and looked more like high-level illustrations than storyboards. The STUDIO does this mainly for agencies whose customers don’t get the ideas they are pitching to them. “Their clients are not very visual so they need to see a very specific visualization of what ouz client wants to present to them. So rough sketches won’t do.” Additionally the STUDIO produces CG-animatics for the same purpose that look like what some cheap productions sell as “finished project”.

But what was even more eye-opening than the quality of the storyboards, was the time frame the STUDIO usually has for such things. “It’s not uncommon that the client calls at four in the afternoon and needs some 12-odd boards with a complicated concept finished by noon the next day. And we can do that. But you have to be very fast. If you spend an hour or more on a single board you won’t finish.”
The strength of the STUDIO emerges from it having “insanely versatile” employed artists available all the time, also on weekends. To pull of the feat of producing a 30-second CG-animated animatic within a week are motion-captured animations and around 30 different rigged characters they can only modify a little. “The Sparrow” for example, with character development, animatic and edit was produced in no more than three days.

“In the US, everything gets tested,” sometimes your creativity is limited quite a bit by the client who tend to cling to the animatic once they’re happy with it. They see it almost like a casting. It can even be arbitrary things they like in it and want to keep, like “I want exactly that dog!” or “I like her shoes!” On the other hand even best and most interesting idea sometimes won’t do it because “they’re testing it against a Midwestern housewives.”
“How can you still be creative in a situation like that?” one question was asked from the audience. Mary agreed and said that it sometimes was hard. “There are different kinds of artists, some need to be creative and others are more crafty. For the creative such projects can be very tough, so we try to delegate the work as best as we can.”

As Known As

Before the next presentation began, the room notably filled with people and having a seat in the front now was worth twice as much and you could tell why. Studio AKA‘s Philip Hunt was getting ready for his lecture on the studio’s experience with getting From Pitch to Screen.

How does everything begin? Basically with drawings. Lots and lots of drawings. They are the fastest thing to do (“The pencil is our most important tool), to shift from one thing to another and throw out a lot of sketches that don’t fit or they don’t like. “I’d say 70 to 80% is immediately discarded.” Then the storyboard phase begins and especially for the clients the 2D designs of characters are modeled in 3D because it is easier for them to visualize them in the final product. But many projects don’t make it to the end and eventually get killed after weeks of work. “You need to cope with that. Sometimes it’s hard. Sometimes ideas get accepted and everything happens very fast,” as it was the case with their commercial for SingUp, an organization helping kids find their voice.

Studio AKA‘s animation for the British Lottery, “The Big Win” set them into a certain direction and other clients wanted a similar style for their own commercials, such as Lloyds TSB, who became “our benefactor bank”, because the communication with the client is very fruitful and they quickly understand from small drawings what the idea is. The studio produced a number of animations for them, although “it is sometimes not easy coming up with a fresh idea after dozens of spots.”

To escape the colorful, friendly and maybe even a bit boring visual world of Lloyds, Studio AKA grabs every chance they get to do something different like BBC’s openerfor the Olympics, who wanted “something like SinCity only with sports.”

And less is often more, like the Love Sports films, whose characters are just some colored blocks. “Character is not about technical stuff or much defined by appearance, its what communicates and connects with the audience and tells a story. You can do so much with so little.

Studio AKA is mainly concerned with commercials, but they also made a name for themselves with their short films and more stylized animations, such as A Morning Stroll that he would present twards the end of his presentation. We all were looking forward to it. “We’re not so much an FX animation studio, as we’re focused on character and narration.” Currently in production is The Beast which is about a beast that lives in the basement and is visually much more experimental as well.

Lost and Found

Working on films is entirely different. They produce them over years and work on it when they have a break from their commercial work, which can sometimes mean a few days, but also not being able to get any work down for weeks. The film that was shown also shown at last year’s fmx as part of the Shelly’s Eye Candy screening. I remember not liking the sequence on the rough sea very well. I found it to be too long and that the appearance of the ocean clashed with the style.

“We really liked the book but we had to expand the story much to make it work. So we did a version of the book, not the book.” In the eleven months the project took place, the creators learned “the hard way about pipeline and planning.” A thing they prospered from was the involvement of Oliver, the author who brought himself in and enjoyed working and redrawing on his original story and set drawings and even small text for signs from New York via email.

The look of the ocean was a challenge of its own and one person worked nine months to get it right. The mass penguins on the South Pole were achieved by crowd animation but unfortunately this produced some “possessed penguins” whose animation was messed up and they dithered like they were, well, possessed. “I tell you, animating every last penguin by hand would have been less work than painting out those bastards!” Philip explained their solution to the problem.

They also got a complaint by the safety department why a kid was without a life-jacket and a guardian making his way alone to the South Pole. “So I explained to him that the trip was a dream the kid had, the rough sea part being a nightmare. And about the octopus,” he turned to the audience, “Do you remember when you were a kid and had a bad dream and fell out of your bed and then your parents would come and pick you up, you were still half asleep, and put you back in the bed and tug you in? That’s why the kid is dreaming of the octopus.” That really sounded like a reasonable explanation. “They bought it!”

One person worked really long on an automated and very complicated rig for the arms on the octopus. When the time came to animate it, all it would was wobble its arms, something that was not in the film at any point. So one of Philip’s colleagues looked at the storyboards what exactly needed to happen, considered it for a bit and said, “Yep, give me a week and I can do it.” And he did. “It’s a good example of what results blind panic can achieve!” Philip concluded.

Then it was time to screen A Morning Stroll to us, the first public screening of it — yay! And it really was everything that I was hoping for: Funny, witty, experimental perhaps, and with a great twist at the end (that it wouldn’t quite need in my opinion though). In the animation there even was a fictitious iPhone game featured “and we’re trying to make that game” Philip closed. Some still chuckled from the animation.

MacGuffed

Even more people flooded in despite it being the last day of the fmx while Pierre Coffin from Illumination Entertainment was setting up his laptop for the presentation. He looked into the audience, the room was completely full. “You all came to see this? You people should get a life!” before he began with outlining The Making of “Despicable Me”.

Originally Coffin was approached by an US producer with some concept drawing and a rough idea he wanted to see in a big film. It were drawings of a villain in everyday situations, how he would live his day-to-day life. “He really wanted to get the project done, so we went to dinner, I had chicken which was really really good and he presented me some ARGUMENT$. And then more, he was quite A$$ERTIVE.” Coffin declined until he was promised to work with Chris Renaud as his co-director who had worked as a storyboard artist at Blue Sky and co-directed there a Scrat short.

Still, the story was non-existant it was just that idea and a couple of gangs, but “they were strong”. Concept artists drew upon this some more art and coneptpaintings of how such a person would live. It was all very dark and black and steampuk-ish. Some elements from these illustrations were built upon to develop a story, because there just was no script to begin with.

But then, the steampunk-look was toned down a notch because the production grew a little worried to lose some audience, because “it needed to be a family movie. And who in the family decides what movie to watch? Dads don’t decide. The kids do. But only when their mother agrees, the family will watch that movie. So our focus shifted from teenagers to kids and the villain became just some sort of grumpy guy.” Also during the concept phase it was clear, that the protagonist shouln’t be too successful in what he does. “He should be smpathetic. We like failure”.

The actual script came in late: In the last four months of the production the third act got written, not quite knowing where the story was headed. “And the ending came, well, in the end.” Pierre confessed. As the script progressed and changed, so did the animation. “In the end we more or less has to do everything at once”, also (re-)recordings of Steve Carell.

Gru

“We always had this tall, Transylvanian looking guy as our villain quite long in our heads as protagonist.” Pierre continued. The character had already benn built and posed with early models of the girls. This main character had also a clumsy Igor-like assistant who was much rounder and bell shaped. “In the end we decided against him because it just was too much stereotype. And the Igor-character we called Kyle looked friendlier” and he also looked better in the expresson and stating tests. The only thing that needed to be changed was his bell-shape, so they put his head in his torso a bit straighter and gave him the tall long legs of the Transylvanian looking guy. Then animatoros experimented with walkcycles and little stories. “What animators love most is a character playing a charactor, like Gru explaining his money problems to the minions.”

Girls

The girls really had a great design and Pierre was looking forward to seeing them modeld but unfortunately they looked totally unpleasant. Inly two month before the principal animation began they needed to be re-designed and recosidere. The three girls were very long though of as being just one character split into three people but, obviously, this wouldn’t work. So the small one became the true one, the middle one, they almost mute, boyish one and the oldest the reasonable one, “but boring. I know I shouldn’t be bashing on her but I never really liked her” Pierre said. The modeling was done and changed until shortly before the production, “in the last moment we pulled her mouth down for aesthetic reasons with messed with the mapping a bit.”

Minions

In the original concepts the minions were various kinds of baddies but the budget didn’t allow for esigningk, rigging and implementing them, so their design was thought over. The idea behind them was like little plumbers working for Grro. The simpler the shape, the more of them the production could afford and Pierre wanted to put them just everywhere. And as many thing, their final design came “super-late in the production”. The three Minions who get sent off to buy a toy from the drugstore ha a whole story arch around them, like they would get lost and travelled the world unintentionally only to come back at the end to save the movie in the most funny way, “but we never could think of something good enough. So we discarded it.”

A typical shot

Everything would start from a scene in the scrip or a recording of Steve (who also came up with Gru’s accent and the names for the minions) for the storyboard.

Pierre’s main direction to the animators was that he hated overacting and “illustrative physics” as he alled it, something quite different from DreamWorks. He also wanted to get to the point visually very quickly without just stating the obvious. Another philosophy was to get to get to the idea, to the core of something, instead of a very elaborate storyboard, for example. The storyboards sometimes only showed generic locations, axial jumps but, again “that is not the point of an animation storyboard.” Based on that, the set and the layout of the scene were designed and modeled, everything still very rough and just to make it possible to point out blocking problems and camera angle issues.

“There is this myth of putting all and any of an actor’s performance into the character no matter what” Pierre explained as he was showing one of Steve’s recording sessions. “Overall he’s not really expressive all the time.” The artists sometimes filmed themselves and intercut the footage (like it had been donw occasionally on TRON Legacy with Jeff Bridges’ performance captures) and used that as a reference for their animation. Then they made blocking tests, just the keyframes of their animation to see whether it would work inthe framing and the set. Once this blocking pass was greenlighted the shot could evolve further. In parallel artists color sketched the scenes for the lighting artists. As said above, everything “kinda happened at once”.

Even despite the fact that the movie came out in stereo, it was not a particular technical challenge. “We did it in 3D because they wanted it and we made it work, that was it. But we didn’T invest months of heavy-duty-research like Disney on the use of stereo-3D” Pierre summed up and showed a very funny little short animation of the minions, titled Banana. We did a couple of them, it was like a TV animation and didn’t spend more than two to three weeks on each. On the feautre we worked three years in total.”

Chaos Theory

Then it was noon again and again I dropped my wallet, but this time I found it within minutes because I already expected it. Sounds like I should change something. Maybe I need more credit cards or something. So a strawberry smoothie and a box of rice later I was ready for some more knowledge. I just wish I had hurried up a bit, so I got a rather bad seat. Pixar’s lectures are always insanely popular, no matter what. If they just would present a new RenderMan version, everybody would be over the place. Wait a minute, that was the case three years ago!

This year again Variety‘s David Cohen said a couple of introductionary words and let Bill Polson begin with his lecture with the interesting title Chaos Theory: Making More than One Movie at the Same Time”. The first thing he threw on the wall was a roster of the release year and title of each Pixar films so far that showed that Pixar more or less released a feature every year in recent years.

Oh Shit!

But Pixar works four years on a movie, and those four years are split up into four stages, Bill lined out, Preproduction, Pipelining, Modeling and Shots, “or as I am going to call them Oh Shit!, Chaos, Stability and Crunch.

In your Oh Shit! stage you have no idea where it’s going and how to do it, the Chaos stage is defined by developing software, getting things to work and figuring out a way to put everything together. Stability means that everything is working and everybody knows what to do and where the project is going, and finally< in Crunch you pull over-nighters, some get carpal-tunnel syndrome and you just try to get the thing out.”

Then he gave some examples of the Oh Shit! moments with the corresponding Chaos stages in the last few productions, a slide I will reproduce here:

The Oh Shit! stage in Cars was figuring out how to make and animate cars and especially reflections; Ratatouille posed the problems of clothes, fur and rendering believable food; WALL•E asked for expressive robots and a world of trash and Up was set in the jungle — “The more things you do, the more things are going on.” Moreover these new additions to the shading pipeline needed to be implemented without braking what was already working. Pixar always tests new techniques against the models of the previous film(s) in order to ensure consitency and compability. So the shading pipeline of the last years looked like this:

So in 2007, for example, there were four different shading pipelines at play. In context of all the Pixar films, is has been that was since 2003.

Impact

Next up Bill showed how multiple films-overlaps at different stages overlapping affected the crew and culture, software development and the studio itself.

“Let’s see how the situation when we release one film every two years.” In this scenario a lighter would be in Crunch mode on a feature, the next year there would be nothing for her to do, only to join the following year another movie in Crunch. Since Pixar’s philosophy is to have the artists on staff at all time to ensure they share the same culture, they can’t have their lighters unoccupied for a year, so “they are encouraged do modeling work for the next movie.”
In an environment of releasing a film every year, lighting artists can either specialize in lighting or skip film, the latter not being encouraged because “when people skip every other film, their culture becomes fractured and this slowly adds up to an environment where people start losing touch with each other. A film means learning. Miss a film is missing an opportunity to learn.”

In terms of modelling and shading this fracture unfortunately inevitably appears because the artists need to be present in both Chaos and Stability stages.

But the worst break is evident for the technical leadership because they accompany a project from start to finish over the whole for years, “which would mean they can only work on a film every four years.”

So what can be done about that situation? The answer is a mix of generalists and specialists. Specialists work on projects of a short release cycle, whereas generalists can work on films with in a longer cycle. Cohesion is maintained only for single-year departments, as soon as multi-year departments come into play they get fractured. Worse, the technical crew gets fragmented and hyper-specialized.

Software

The problem with software is to keep the versions stable over the course of a project. With a film every year the software departments needs to manage and keep track of four to, at worst, eight different versions of software combinations, which is a huge problem. Currently there are two different systems in each screening room, each for a different production, moreover the screening rooms themselves are dedicated to, either project A and B or C and D.

For the software developers the sweet spot is a release cycle of a film every three years: They come to it pre-production, work on the software through the Chaos stage and in the third year stay on the project until it reaches Stability. When the productions shifts into Crunch, the developers can get to a new project.

But in a two-year cycle this looks differently: In the stage they would be working on the stability of a software for one film, they instead are on a new project in the Oh Shit! stage already, which effectively means that they leave the software before it is stable. And fracturing developers is a bad thing, but in order to get things done, you need to have 2 pipelines.

Bill put up another table: “When you as a developer join a new project in the Oh Shit! stage you need to answer a few questions about the status of the software of the film you were previously working on. This works out pretty well, if your previous film is in either Crunch or Stability,” as outlined just before:

So if the Oh Shit! moment occurs during the previous film’s Stability or Crunch stage, you’ll likely be okay.

So what they want in a yearly release cycle is the software developers jump from Stability to Stability stage — instead they get them jumping from Chaos to Chaos:

That paints a pretty bleak picture: If the Oh Shit! moment occurs too soon, when the previous film is in Chaos, “you’ll likely whipsaw the pipeline and your crew”. Bad. Very bad. “Part of a solution is to only let go of the previous film once everything works. Otherwise there is nothing to test the new film against and you’re flying blind for two years. So getting the stuff working is critical.”

So what are answers, rather, are there solutions to the problems? Bill sent a last table on the big screen:

“There’s a thing to say about the specialists vs. generalists” Bill said as he touched the subject of what kind of Pixar is interviewing and eventually hiring. “Schools that force students into specialization in their last year have a much better chance to get hired by Pixar,” as compared to students who come from schools that train generalists.

An Answer?

It seemed to me that there were not many (if any) solutions the situations and the slide of the presentation was rather general in the solutions it presented: “Management, trust, etc”, “Department structures”, “Focus on stability rather than artistic reach”, and “Others ?”.

In the near future Pixar will be producing three films in two years, so all the tensions and problems will only increase, “so we got to standardize” which was difficult so far. Lucky for Bill, who is concerned with the more technical task of pipelining and management, getting the stories together takes currently longer. “The lack of good stories is the main thing that has kept us from scaling up” so having even more movies in production at the same time is currently not an issue. “Once they get solved the technical stuff is not ready yet. But we’re working on it.”

In Pixar’s early early days there only were generalists around, people who also flowed in an out between technical and artistic departments and groups. “Now the walls are high and you need to specialize. It was more fun in the old days” Bill reminisced. Still, Pixar’s philosophy is to get motivated and talented people and make them great.

Q & A

“And now I would love to hear your war stories” Bill finished and sat down in a chair next to David Cohen on the stage.

“How does the short film division fit in?” one guy in the audience wanted to know. Clever question! Bill explained that they had to slice up the production to fit the shorts in whenever they can to fill the gaps in the rolling production of features. “The difficulty is to have the inventory ready”, because (much like what I heard about Studio AKA that day) there are months when nothing is done on a short, then a couple of animators work like crazy to finish in two weeks. Again, the short hibernates until the lighting artists have a couple of days to spare. “It’s incredibly difficult to schedule so we now treat and plan shorts like miniature films.” Shorts are also a good way of keeping the artists motivated and help them getting out of a rut.

“(Why) do you need to change the pipeline every year? Is this really necessary?” Bill explained, that every film need things implemented, that the previous films did not need. “Between WALL•E and Ratatouille we changed the definition of the geometry and shaders, lighting and added post effects. This in a way broke the ability to test against the old productions and the production few blind for two years.” Also, there are a lot of dependencies that get affected when you change one thing, say the shading affects also lighting and rendering. So Pixar changes only a few things at a time, keeps things as stable as possible and still, those petty paced steps end up in a pipleline that has not much in common with the one from five years ago.

“What do you do to maintain the culture, if it still keeps getting fractured so easily?” Pixar is assembled a groups of superiors who “own” teams of artists, like all the animators or all the lighters. This group also indicated community-events such as training for their employees, cross-show lunches or the monthly lighting-lunch where artists show, across the various projects, what shots they lit and get feedback from other artists. Further, there is a newsletter to keep everybody on the same level of information.

“How does standardization compromise creativity?”, a rather opinionated question arose. Bill smiled “In fact it does just the opposite.” He told the story, that even the shorts needed an adjusted and new pipeline to work. However, One-Man Band used the pipeline from The Incredibles without any adjustment because it didn’t need any. “It all worked, it was cheaper, it was faster and the animators had a great time” because everything worked like it should right away. On a related note, Autodesk organizes bi-annual meetings among the industry leaders to talk about what they want in future Autodesk releases, “but you don’t talk only about Autodesk products”, Bill assured and explained that in those meetings certain standardizations for core elements originated such as Alembic.

The Pixar Brain Trust is a small group of creative leaders at Pixar who oversee development on all movies. The group came about during the development of Toy Story. They meet frequently to watch the status of films currently in production and tell the directors their criticism. The directors have to listen to the Brain Trust, but are free to ignore their opinions (although this is rarely advisable). There also exists a Technical Brain Trust at Pixar who bring in their ideas to the Technical Supervisor who is also free to ignore the suggestions, yet implementing suggestions “its different with the Technical Brain Trust because the changes affect multiple films.”

“What if I have a real good idea for a story?” — Pixar doesn’t acquire stories, simple as that. Instead directors should come up with their story ideas themselves, “we want them to be passionate about their stories.” When the very first idea for Pixar-stories are thrown in for discussion by the director they usually are not the that good in first place,but have potential. “If you give a mediocre story to the best people, they can turn it into something great. Give a great story to mediocre people, the final result can only become mediocre” Bill explained. And basically, anyone at Pixar become a director. Usually people who have been observed being skillful in story and art get asked if they wanted to direct a film. If they accept, they are given the time to come up with some ideas for stories, which can be as simple as “What if a rat wanted to become a cook?”. Over time these stories get better and further developed until they are ready to be produced.

This culture of participation among all Pixar employees is what is unique to Pixar. There are oftentimes organized screenings of the films in production among the Pixar employees, from directors, to artists to security guards to kitchen staff. Everyone is encouraged to send notes and ideas to the producer like “I didn’t get that joke” or “that scene was boring”. This openness, it seems, served the integrity of Pixar’s storytelling well.

Layout

Next up was another Pixar presentation and again, the König-Karl Halle was packed. In all the chaos of people leaving and others coming in, I was able to catch a seat up front that left me more room to breathe and, more importantly, to take notes.

German Filmakademie Badem-Württemberg alumni Saschka Unseld and layout artist at the studio with the lamp was presenting Cinematography at Pixar. He was introduced by Terrence Masson who I observed tirelessly swiping away on his iPad the days before². Saschka’s presentation also featured a sneak peak of Cars 2 but mainly dealt with the opening sequence of Toy Story 3.

“When I get asked where I work the immediate response is: oh you’re an animator then? When I tell people that I am a layout artist they usually go ‘huh?’” Saschka laid out the situation of layout artists. Layout is all about camera, staging and cinematography, he summarized, “or visual storytelling.” Pixar employs 20 to 25 layout artists of whom 12 to 15 people work on the same feature together.

In principle the layout artist gets the storyboard(s) of a shot, creates very roughly the animation and interprets and explores it via camera angle, position and movement. This means for every single shot the artist tries out a number of different angles to give the editorial department plenty of stuff to work with. “It is not about recreating the storyboard as close as possible but to use it as a guide and to express its story point visually as good as possible.”

This starts with animating said sequence as a blocking pass and then putting cameras into the scene. An exchange between art and set-creation is always happening, for example when there is not enough space for the action to take place or elements obstructing it. Of course, there are some per-shot changes in the final film, but “we try to avoid it as long as possible.” in case you can’t tell: At Pixar work perfectionists.

Just like on a real set the action is filmed from different angles and in dialog scenes, there also are master shots rendered for the editorial department to cut together to their liking. The editors may also occasionally retime shots to make them work. When their work is finished, the shots come back to layout where the retimed shots are re-animated in accordance to the changes from editorial.

An important factor of this work is “Shot Hygiene” which means that every file that leaves the layout department must be clearly built and named, cleaned up, properly linked and have the exact frame range because the files files then are given to the animators who place their work right in the file, and so does the lighting department, for example. “When it leaves layout, the film is done.”

With every feature there also come certain cinematographic concepts and principle, one applying to all Pixar films: “Restrict yourself!” just because a camera can fly around everywhere and do crazy stuff, it shouldn’t. I guess we all have seen amateur works with nauseating and impossible camera movements and that it what they want to avoid, to “feel” CG. Many such real-worked developments were introduced in WALL•E as Danielle Feinberg explained two years earlier. So in addition to having a specific lens-set (ranging from 10mm to 150mm), the use of certain lenses in Toy Story 3 was restricted to either toy perspective or human world. Since the world of the toys should not feel too small, the DoF was kept high to avoid a macro look like in Toy Story (1)³ and to have the human world and the toy world consistent.

But staging is also a matter of framing as well. A character’s high point can be emphasized by putting him or her really on top of everything in the framing, in the low-point the arrangement and composition might be weighing down on the character and isolating him or her ⁴. You can also imprison your characters when they are trapped also visually and so on. Again, layout is visual storytellling and the link between the story department and animation.

Sequence Evolution

How does not a sequence evolve from a story board to the finished and locked layout? Sashka laid this out (pun intended) in detail on taking the opening sequence of Toy Story 3 as an example, were everything started out with the “Set Scout”> where the script and storyboards of a sequence are reviewed by a artists from layout, lighting set-design with the DP.
In the “Location Scout” phase the art department roughly designs the set on a plan where director and DP often propose changes. This very early and very basic set model then gets blocked out by producing a number of still frames to check the proportions and sized of the characters against their environment. The results are presented to director and DP for feedback and changes are made in accordance to their feedback once more.

“Shot Blocking” follows, the actual layout stage. Again, the angles from the storyboard are not simply copied, the blocking must tell the story beats as good and clear as possible. Sometimes it is possible to combine two or more story boards, at other times, the action needs to be broken down in more shots than anticipated. Since there is a lot of exploration happening, a layout artist might not have all the assets she or he needs for an idea, “if you don’t have something you want, temp-in something” Saschka encouraged.

What camera angles and shots end up on the big screen gets decided in the editorial with director and DP. Layout supplies them with as many interesting ideas to chose from for each beat as possible.

In the end, Saschka presented a short sequence from another feature’s opening sequence, Cars 2 where he showed the evolution of an action-laden scene from storyboard to final. It was stunning, how visual ideas Pixar managed to pack into it. “Are you sometimes not satisfied when you seen the final?” Saschka was asked as the clips had finished. “Always,” he answered instantly “every time I watch it I find something that I could have made better. The longer you explore a set, a shot, a beat, the better your ideas get” he concluded.

My mentor and former Disney animator Fraser MacLean is finishing his book about The Art & Evolution of Animation Layout which will be out in August this year. Highly recommended!

Megacity

The audience’s transition from Pixar to DreamWorks took a little longer and I found myself sitting next to some American college students, as it seemed to me. Almost instantly after the lights dimmed down and Philippe Denis began on “Megamind” — The Creative Process, the one next to me whipped out her Blackberry and launched Texas Hold’em Poker, her other hand found yet another cell in her purse via which she was heavily involved in keeping up what was new and cool on Facebook — the entire presentation. I really am thankful that my social life and craving for distraction are not as demanding, so I was able to take quite a few notes.

Without losing too many words about it Philippe verbally rolled up his sleeves and got to work, displaying an abstract slide with circles and lines on a jagged grey shape. It was the basic street layout of Metro City. Just like Blue Sky for Rio, they came up with a procedural approach for generating the city. Based on the rough boundaries and boulevards laid out beforehand, the tool created the street grid, then populated it logically with blocks and lots. “And in the lots the buildings could get placed.”
On the city map different colors indicated different types of buildings, residential lots were red, commercial ones where blue⁵ and so on. To control the height of the buildings a simple height-map was plugged into the script that would select a building type and then scale it to the desired height.

“Scale to the desired height?!” I thought but of course there wasn’t mundane scaling at play. Instead, the different types of buildings were coded, so that scaling the height effectively meant that more floors would be added automatically between ground floor and rooftop, each cleverly broken with ledges and other architectural braids. “Architecture is all about proportions” Philippe added as he rolled a screen recording from inside Maya where the operator scaled such a “coded” building into every direction.

And there were a lot of buildings to design and code. Ultimately the city was composed out of 20 thousand people, many thousand cars (following a traffic rule-set) and 70 thousand buildings, each with different LoDs and corresponding maps: If a building had a certain distance to the camera its modeled architectural details would retreat into a normal map, for example.

Also, the roads and streets were designed and coded in a similar manner, even details such as crumbled curb-edges, mailboxes, street lamps etc. had been modeled and were placed by the algorithm.

But the city’s buildings also needed to be shaded and this was where things started weighing down in memory; the whole city was around 1 terabyte of data. The solution was to bake the complex shading networks which resulted in a seven times faster rendering.

The city, however, would not only be used as a rigid stage, parts of it would also get destroyed at some point in the movie. As for the FX development there was a close cooperation with the other departments to always keep the direction transparent for each artist. The dust blown up by collapsing buildings was, for example, realized with Maya fluids that interacted with the geometry, so even DreamWorks put their pants one one leg at a time.

“A Cape Doesn’t Make a Hero”

“We also had to realize some character effects and because Megamind is all about superheroes it’s mainly about capes.” Philippe started off and displayed some cloth simulations in which the cape didn’t behave like it should, or to be more precise, like the animator wants it to behave. To cut a long story short the solution was giving the animators tools to deform and pose the cape with IK/FK handles; curl, skew and sine wave controllers and to also simulate the cape. Afterward the simulation could be seamlessly mixed together with the animation.

City Lights

In terms of lighting DreamWorks settled for a new path which seemed old hat to me to be frank: Philippe showed a shot board of some of the feature’s shots turned into black an white. It was obvious that a bit contrast was missing. Proudly Philippe beamed a slide-filling tone curve that was ever so slightly S-shaped and top-heavy. This evidently increased the contrast of the final renders and also provided a gentler roll-off towards the highlights.

But also the exposure range was presented as a bit braver than in a traditional animation as it had been decided to let things blow out into overexposure when they were not necessary to the shot (“we expose for the character”), such as bright-lit buildings in the background when a character was standing in the shadows in the foreground. Also nothing really new.

Metro City at night on the other hand asked for a little more ingenuity. I guess some of us had to build and mainly shade nightly illuminated CG-buildings. As soon as the camera starts moving (or in stereo 3D) you won’t get away with incandescent “interior views” that have been plastered over the windows. That is why behind the lit windows the buildings really had modeled rooms (= boxes) with HDI textures to suggest real buildings. Also the rooms were spanned across multiple windows “which really conveyed a real feeling.”

The street lighting was a bit trickier. The indirect illumination from the streets up the buildings was realized by an ingenious application of ambient occlusion: Take your ordinary ambient occlusion with a large radius, the subtract another ambient occlusion rendering with a smaller radius, tint it yellow and there you have your street-light illumination on the buildings, which worked really well⁶. Still, even by faking it that way, there still was enough to render, thousands of streetlamps with nine light sources each could only be rendered by calculating point clouds for the illumination.

As if that was not enough, a scene set at night also asked for pouring rain which amped up the render time per frame to a buzzing 38 hours, but that gain was feasible since the rest usually only rendered five to six hours per frame: “The complexity was not so much the geometry but the amount of map data and point clouds the renderer needed to access” Philippe summed up.

Like the old days

The college kids with the smart phone stood up as the lights brightened and I was thus granted a little more room in my row. I was rather exhausted of four days of taking notes and typing them into (somewhat) meaningful sentences that I almost forgot that there was still one presentation imminent: Animating “Tangled” by Clay Kaytis, Animation Supervisor at Disney.

“Tangled was Walt Disney Animation Studios’ 50th movie. And the really wanted to make it something special, something that could hold up to the old classics like Pinocchio or Beauty and the Beast in your DVD shelf,” talk about setting the bar high for yourself!

“But everything starts somewhere and often it is not pretty!” Clay smiled and showed the first test of a CG model, animated with blendshapes in Maya. And they really were producing disgusting holes and errors in the geometry; additionally the deformation looked eerie, like a corpse (and I am not talking about the flat shading). “The needs of the performer should drive the design” Clay remarked and went on to a photo of a beautifully sculpted maquette of Rapunzel that got scanned and cleaned. But unfortunately also this model did not work properly with the facial animation system back then.

In about a week an artist took the task of creating a new facial rig to heart and presented Clay with the result on an already modeled character: Instead of blendshapes he had settled for a rig of intricately weighted joints, eight lip controls, six for each eye, four or five for each brow and so on. Even in that early stage it was clear this was the way to go, because animating a character’s face was more intuitive than before. The rig was extensively tested with extremes and there things started to feel a tad too fleshy since the rig did not treat the bone structure differently. To tackle this issue, six extreme poses were defined, drawn and modeled and used as targets, once a certain constellation of joints was closing in on an extreme pose. This method assured a somewhat believable bone structure and additionally art-directed extreme poses. “You want your rig to be like a sports car,” Clay explained, “intuitive, elegant and very responsive.”

For body testing the team really had to gear up as Clay one day slipped that he wanted Tangled to feature the best character animation they had ever done. So they started learning animation from scratch: They animated Rapunzel turning, walking, jumping, looking and so on. Once the animator gave the character a motivation to turn or to act in a certain way, Rapunzel suddenly began to feel like a real quirky teenager, “You need to find a way to have the characters act from inside out.”

The character of Flynn should be sporty and slick although not too grossly “slick”, but the fears were allayed as soon as the character was modeled. Since Flynn and the Horse also needed to be in the sneak peek of the movie, the development on Rapunzel stopped, though.

Tangled did not have dedicated character supervisors. In the beginning all the animators played around with the characters and soon certain animators made certain characters the best.

Glen Keane, legendary Disney animator who had the luck of being trained by the “Nine Old Men,” came initially as a co-director to the production but for health reasons had to leave for six months. When he returned he stepped down from directing to just animating, “a dream come true for us other animators so he would sit with us all day and we could learn from him,” Clay remembered revelling. When he came back after his leave, the artists showed him what they got and Glen started drawing over the viewport rendering. And in 2D he drew the changes needed to be made on the model “which freaked out the animation department.”

In the end there were eight months time to animate the movie when all main characters were rigged, approved and ready for production. “The rig was so robust that there was nothing you couldn’t do with a character.” It was also scripted so it could be set up easily for another character.

Dailies with Glen

During dailies the artists showed Glen what they animated and he would give feedback and draw key poses on a Cintiq over their animation in the screening room. On a new shot, Glen would start drawing his suggestions and ideas of posture and poses and the longer an animator would work on the shot, the smaller and smaller the ideas would get. “Every day was like a masterclass in animation. You go in clueless and walk out with all the right answers to make your shot better.”

When the animation on a shot was final, a breathing pass would be animated to match the breathing of the voice actors which was vital in the singing sequences. Another very subtle detail that got added towards the end was the direction and shape of the eyelashes at each key pose, a detail Glen expertly used in all of Rapunzel’s drawings.

“And he’s always getting better. After 35 years he’s still improving!” Clay finished and rolled a reel of showcasing the animators with a sample of their contribution to Tangled — a well rounded conclusion to this year’s fmx. In the Großer Saal across the street followed the screening of the whole feature but with a heavy heart I had to pass.

“See ya in 2012!”

After four days of too little sleep, caffeine-abuse and battles for up-front seats, my body was aching for rest. Not more coffee, just rest. So I ransomed my car from the ridiculously expensive parking garage and rolled back to my hotel. I was asleep before I remember hitting the mattress.

…but it’s not over yet! Soon I will finish writing the other days’ reports, then add lots of pretty pictures and proof-read the whole grammar-abomination thoroughly.

This site is still currently under construction and more will follow soon.
Watch out for falling pixels!

I stumbled between the rows of the darkened hall as some animation’s credits were running and got my favorite seat to watch a couple of really funny Wily E. Coyote-like animation shorts about Mr. Hoppe, trying different approaches get rid of a barrel of atomic waste that always backfire. There were a bunch of lovely ideas involved, so check out the website!

Know the Past, Conquer the Future

Shortly after Eric Ross introduced the founder of Digital Domain before he sold it to director Michael Bay and a bunch of investors, so he was in the business right from the start on the other end of the spectrum, not an artist but a producer and manager. How he felt after being in the industry for thirty year? “Boy, am I tired!”

“I am going to talk about the past of the VFX industry a bit because it is important to know where we all come from.” Scott joined ILM in the middle of the 1980′s when the company was working on “Innerspace”. What surprised him, was that in fact very little was done on the computers. It was the time when companies had in-house teams (such as Pixar) to develop computers for specific tasks, so called transputers. Since Scott originated from a wold of optical printing and the telecine he thought “that wouldn’t it to be wonderful to bring that to the computer?” At ILM, John and his brother Thomas Knoll started developing a program for doing composits on a Macintosh computer, a program that what would eventually become Photoshop ¹. “But rendering was still a big issue and it only got feasible until we got us some Pixar cubes², which were used for the VFX on The Abyss³. And today? There are probably more colleges that provide VFX education than there were people in the VFX industry in the early 1990′s.”

Scott continued that historically not so much is known about the producers and managers behind the artists of the early days, although somebody “fighting off the client so the artists could work” is just as important. Oftentimes, he lamented, the industry is not really that much interested in the business of VFX. “If you wanna be only an artist then cut off your ear, move to southern France, eat cadmium yellow paint and have a good time!” he joked “In our business it don’t mean a thing if it don’t go kaching.” — simple as that.

But he made clear that the business people are out to suck out all an artists creative power, rather to empower them instead by providing infrastructure, time and the opportunity to individually make a living with their craft. “Well managed companies can always pay all of their employees’ hours all the time.” That made me remember the unpaid overtime I oftentimes did.

History

Scott continued painting the big picture, what companies were founded at what time, and what stages they went through, like the first generation VFX divisions that were all created out of the needs of a single project such as Industrial Light and Magic (which spawned Pixar later) was founded for the purpose of creating Star Wars; Douglas Trumbull created the Future General Corporation for Steven Spielberg’s Close Encounters of the Third Kind and Rob Abel founded Robert Abel and Associates (RA&A) for Star Trek. All that happened in the 1970′s.

The second generation facilities were founded mostly by people who were fed up “working for the man” and decided to start their own companies. Rhythm and Hues was founded in 1987 by six former RA&A employees, Richard Edlund’s Boss Films, John Dykstra’s Apogee, Inc. and Scott’s Digital Domain were all companies created from former ILM guys who had a “religious problem” with George Lucas, as Scott put it: “He thought he was god, and I disagreed. So I started my own church.”

The next wave were studio owned VFX divisions that would open and close and open and close, depending on whether they were needed for a production or just were unprofitable such as Buena Vista EFX. Warner Bros. Digital is now closed, Sony Pictures Imageworks costs the studio a lot of money but they produce very well grossing features, or The Secret Lab that used to be Dreamquest got closed after Disney’s Dinosaur bombed.

Of the 3rd, 4th and 5th generation facilities some made it until today, such as Weta Digital or Method Studios whereas many didn’t (The Orphanage, Station X, Asylum, Café FX, etc.) or as Scott put it: “The highway is littered by dead bodies. For a successful business you need to understand the business aspect as well as the artistic and the technological aspect. And still, that might not be good enough.” And a good name is also important. “Somebody back then suggested Digital Domain, whereas I wanted to name the company Presto Digital-tation or something like that. I’m terrible with names… I have three children but I won’t tell you their names” Scott said with a smile.

And today? There are mostly studio-owned companies that produce full CG movies, which turned out to be highly successful: Disney got Pixar (“Toy Story”), Fox owns Blue Sky Studios (“Ice Age”), Dreamworks acquired PDI (“Shrek”) and Sony has Sony Pictures Imageworks that currently delivers high-end VFX. The advantage these companies have, is that they provide the content creators with all what is necessary such as distribution, marketing and licensing. Still, “content is king. Making and owning content will do you right” Scott is convinced.

The crux with today’s high-end VFX and CG production is that it does not happen in a free market environment. “In VFX, the consumer has no vote, s/he can’t say I am going to pay $15 to see a movie by Sony!. Instead the studios control the business, which means that when you have a VFX facility you have these six clients worldwide.”

The thing is, though, that the facilities are not profitable, as said before for a number of reasons: You can’t see long ahead if and when there will be which amount of work. Then, it is difficult to price VFX since everybody is constantly asking for something new that had never been done before which make it impossible to compare. What is really hard on the US-based facilities is the subsidized situation in the UK where VFX companies are basically spared from some taxes by her majesty. The cost of labor has steadily increased (which is at least one good thing for us artists) so when an artist got an annual salary of around $40k for the work on The Abyss the same skill set and expertise earns you today a healthy $200k to $250k per year. Broken pipelines are another issue that might cause great damage, technical prowess, managing the client, next-gen facilities, capital investments and satisfying the management’s expectations — that all costs money.

What grosses money today

The 80′s were dominated by a film star scheme that worked really well: Have a familiar name on the poster and you were certain to get your revenue from it. As Scott showed the top 20 grossing movies of all times there were only three of them starring a movie star with appropriate pay: Johnny Depp. Almost all the others were features heavy on VFX where the lead roles didn’t make a difference, “VFX and animation mean everything today!” Scott tried to motivate us. I know that I had really gotten worried on how the industry is headed right now. “The new film stars are you, your work gets shown in every trailer, not a witty dialogue by Tom Hanks, the images you create people want to see and pay for!”

In the end Scott talked about the Korean film “The Host” that was incredibly successful in Korea and Japan, “but if it was produced in English with a slightly more Western structure in the narrative it could have made twice as much internationally” he concluded and was running out of time on this extensive lecture so he jumped through the importance of outsourcing and spoke a few words of warning, that the low-cost content providers will become the provider of services in the near future — like it or not.

“But there is a bright future for you as the content provider, because, again, content is king!” he said. With the new models of distribution with the internet the current system of the Big Studios can easily change. “There is a bright light at the end of the tunnel. But it could also be the light of a train. You need to find a place to fit in, and don’t think that everything is gonna work out like roses, because sometimes it just does not.” I was irritated by the mixed messages I received from his lecture. And all that without coffee made me feel like Garfield on a Monday morning.

Oh, and here are two interesting facts from the presentation I could not find a place to fit in the text above:

• Scott played the cowbell on one Jimi Hendrix song. Funny story.
• Monkeypoints is net-profit on a movie. Since the studios have clever ways of shuffling and hiding the money a feature makes, it practically means you never see a dime; whereas First Dollar Gross means a percentage of what is made at the box office.

Going Global, Pt. 2

I remained in the König-Karl Halle because I was too exhausted to move and so I sat there and watched the panel that was up next. Eric Roth moderated the discussion Global Production from the U.S. point of view among the illustrious round of industry leaders and professionals. On the podium sat Jeff Okun, VFX Supervisor and chair of the VES; William Sargent, the founder and CEO of Framestore, Dan Glass, Executive VP and GM of Method Studios/CIS and Lee Berger, the President of Rhythm & Hues. I tried to note down their respective views and opinions on the topic.

Lee Berger stated right in the beginning that 90% of the costs is labor and if you want to stay in business you want to cut the costs by retaining the quality. And outsourcing is a way of doing that, in the case of Rhythm & Hues about 30 to 50 percent of a project is worked on abroad. Taking William’s statement “in the end it still comes down to the price” into account, Eric Roth asked if this was going to become a race to the bottom of the price. “Definitely. But you still don’t want to cut any corners quality wise. India and China catch up in quality pretty fast and will soon be able to deliver the same quality. And we do whatever we can to keep the prices low.” As brutal as this may sound, R&H still managed to stay in business for 24 years and counting and always managed to pay their artists.
A question from the audience was asked whether this fragmentation will continue to a level where the artists work on a powerful system from their homes and exchange via the cloud. “That would be too fragmented for productive and creative work,” Lee explained. “It is already happening for tasks like roto where you pay somebody $50 for it, but for creative tasks it just won’t do, you need to be around each other.”

Dan Glass considered taxes and subsidies to be a huge factor for where in the world to produce VFX, but also the skill set teams is important. “We want our teams to be self-sufficient, being able to work on sequences independently. We don’t split up tasks.” Apart from filmmakers still pushing for high quality and not accepting mediocre VFX, he also talked about the development that low-cost labor will not remain low-cost forever, so you always need to be vigilant about the global fluctuations and tendencies.

William Sargent was rather pragmatic in his view as an CEO of a big company, and stated that “in the end it still comes down to the price”. But in his eyes the globalization doesn’t really mean shipping jobs abroad, instead just growing abroad. Framestore also operates an office in New York, “but since New York is the center for commercials we just wanted to be near to our clients there.”
Nowadays you get paid for what you can deliver on a per-project basis, nobody wants to pay for the R&D that goes into the steady creation of something that has never been done before. But a positive development in recent years is that the VFX houses have become part of the discussion. “Clients say: We have $ 5 million and want to do this. Is that feasible? How can you help us achieve this?” In Williams’ opinion, the VFX industry has the best organized and still most flexible part of any production, “things can virtually change over night and we can react to that.”
Eric asked if he gave William $50 million, would he found a VFX facility again?
“There’s definitely an easier way to make a living than to start a VFX facility, but this is just where my passion lies. And with $ 50 million you wouldn’t open a VFX studio, you would start a project instead and maybe found a small VFX studio in the wake.”
But is being a small VFX studio with a better profit percentage preferable to a bloated system? “Well, you need a certain size to being able to obtain and finish some projects.”

Jeff Okun saw the change toward a global industry in bright colors and as an opportunity for the creative: “We are the migrant film workers,” he continued, “I didn’t get into the business to win an Oscar but I love figuring ways out to do the impossible and I don’t hesitate to take my family with me and go where there are opportunities. I lived half a year in New Zealand, six months in South Africa, six months in Thailand… I don’t think geography is that important anymore, it doesn’t matter. The only reason the industry is still so US-centered is because it’s where Hollywood grew and became the center of it, where everything used to be in once place. The US is a sad place right now, a lot of bullshit going on.”
“Embrace the change, change is good. It gives everybody the opportunity to fix things that are broken. This is a good age to be a student and a good opportunity for a do-over.

Fun fact: Bill Kroyer sat in the row next to me, a book with the colorful cover next to him (CG101: A Computer Graphics Industry Reference, 2nd ed.) and I was racking my brains to think of something I always wanted to know about the original TRON because I’d probably never have the chance to ask him again anytime soon if ever. In the end I nodded off during the Q&A session in the end. When I woke up, Bill was gone. Damn!

Outsourced

The next session of that day’s focus on producing global was held by Philippe Gluckman of the DreamWorks Dedicated unit who had started as a supervisor on Shrek and was at PDI when they got acquired by DreamWorks. In his presentation he talked about the struggles, surprises and success of building a DreamWorks animation studio in Bangalore, India. “So basically what we tried was to apply the same process of unification that was going on when PDI merged with DreamWorks.”

In May 2008 the studio in Bangalore was opened in cooperation with Technicolor that already operated from the same tech park. The studio “opened” in a sense that it was lacking employees and equipment but the premise were ready for them.

What was the goal for DreamWorks’ Indian unit? Cheap work? Low expertise work? Philippe made it clear that was not the case. “We wanted to make it the equivalent of PDI with the same level of quality.” Then he showed the studio’s demo reel with various Madagascar or Shrek themed seasonal TV specials. To me it looked just like the movies. “The only thing that’s different so far is that we don’t make or own innovations, we just use the techniques that PDI already acquired.” But what the Bangalore studio does not do is recycle already used sets, because it doesn’t quite work and in the end you spend more time making an existing set work in a new context than building a new one altogether.

Training

“We needed to train the local artists with our software first. It is proprietary and different than what the artists are used to. It has no full documentation and takes quite a time to get comfortable with.” But the training also needed to be broader than just teaching the tools. Some also needed to brush up their English and the leads were sent to the US, just as some people from the US were sent to India. “Our artists really enjoyed those visits and loved to be exposed to the top people of their field, they wanted to soak up their knowledge like sponges.”

Culture

What about the cultural differences? “The people weren’t used to the freedom we gave them: Instead of urging the animators to churn out a shot each day we gave them six weeks for it. But what sounds like creative heaven at first also needs to be seen in perspective: You need to figure out a way of working that fits you and that you are able to constantly improve on your shot. To keep this level of quality we ask much of our artists but they reward us every time.”
“At PDI there’s a culture of voicing you opinion and exchange so you also need to see criticism as opportunities, not threats. In the beginning it was difficult to get the artists voice their opinion and to be as forthcoming as would like them to be and pitch their shots before others. They got used to Tell me what you want and I do it exactly as you say. That is not how we do things.

The time difference of twelve hours soon made it obvious that there were only limited opportunities for reviews from DreamWorks in California. Since the studio felt like a start-up, sometimes people got promoted to fast into leading positions they did not feel quite ready for.

Technique

Technically there also were some challenges waiting to be mastered: “There was no person in India that lights the way we do. We don’t have a compositing department, everything happens in the renderer so we really had to train people first.”

Also the matte paintings in the trademark DreamWorks style were too uncommon to find a talented artist right away since they are not photo-real but much more detailed than a concept painting, they lie someplace in between.

In practical terms also the look of some elements resulted in different comprehensions, as with pixie dust. “Magic is terribly hard to get visualized because it is highly subjective compared to, say, water.” The Bangalore studio also animated clips for a DVD menu that asked for a fully deformable yet still fully loopable fire, which posed “a tricky but interesting task.”

The Future

DreamWorks’ goal is to have a studio in India that is capable of developing and producing a full feature with the quality standard of the headquarter in the US. “And this happens right now in Bangalore.” Philippe closed before showing some personal photos he took of the people he met on his exploring walks around the tech-park.

WALL-@

Again my caffeine batteries were empty, my low biological activity almost drained as well and it was hard keeping my mental focus. And to my knowledge there is only one medicine⁴: COFFEE! So I tried to get out and to the other Starbucks as fast as possible to avoid long queues. I finally arrived, merrily ordered a salami bagel and a caramel macchiato and reached for my wallet. It wasn’t there anymore. I looked incredibly stupid at the barista “I lost my wallet.” I uttered. This was not happening, I thought, there was everything in it! “Tough luck. NEXT!” he replied and I traced back my steps. Vanished was my exhaustion, exchanged for a cold panic with sprinkles of irrationality that made me text my beloved Conny who was hundreds of kilometers away and couldn’t do anything about the situation apart from sending a reply: “Oh no!!!!”. Back in the König-Karl-Halle at my seat there was nothing. I asked the technicians, they didn’t have a clue. I went to the info-desk and ask whether a wallet had been found. “Yes. What name?”. I showed her my ticket and she handed me my wallet. Even the money was still in it! I was so relieved I babbled like a madman about how relieved I was and sent an all-clear-message off to Conny. I went back to the coffee shop with a fresh boost of relief-induced energy and placed the same order at the same barista. “I hope you can now pay for it” he replied with a smirk.

Why did I tell you this? Because you should learn from my mistakes: Don’t hog so much loose change that your wallet gets so heavy, it falls out of your back-pocket when you try for three hours not to fall off a chair.

Forged Cutting Edge

FMX flags,
originally uploaded by Phil Strahl.

Usually I would have jumped into the conversation of the fellow fmx-attendees but my internal computer said “No!” because it runs on caffeine and was dangerously low on it. So as soon as I got downtown I went to Dr. Starbuck for my medicine before attending the day’s first lecture.

Peter Plantec dropped a few words about today’s special program on virtual humans, after all he had written a book on the topic. It’s almost safe to say that finally we have made our way out of the uncanny valley. Almost. There still are occasional outliers but Benjamin Button proved, that it could be done.

And Digital Domain was spearheading that development as Matthias Wittmann’s presentation Combining Ages — TRON Legacy showed where he talked about the development and realization of Jeff Bridges’ digital age transformation on screen.

Head of Development

So how do you start such an endeavor? First you make a physical life-cast of your actor which gets you a full cast of the head (also the back of the head), good detail of the wrinkles, a sense of the underlying bone-structure and position of the ear holes, all that won’t change over the years — as compared to sagging skin. The downside of this method is that the person has to sit still for half an hour. “And when you sit still for so long you relax and your jaw drops. Like in many of you right now, your teeth-rows aren’t touching. And that’s a problem when you want a perfect model.”

Another time Jeff’s face was reproduced, that time it was a LightStage scan of his face. “LightStage has a really high resolution but you have to sit perfectly still for five seconds and look into bright lights. So inevitably your eyebrows narrow and even very subtle movements during the scan distort the geometry a bit.” But they already had the brows right from the life cast, so the artists could combine it with the jaw from the LightStage scan and a modeler crafted a clean digital model in Mudbox with proper topology.

Now they had the old-Jeff head. This was needed because the solver of the tracked footage recorded from today’s Jeff Bridges with four head-cams on set would only deliver correct results if applied to the old-Jeff model which would drive the animation on the young-Jeff head.

This head was modeled after the scanned geometry and countless reference photos, movies and videos depicting a young Jeff Bridges. This young-Jeff model went through countless back-and-forth stages of “Are the wrinkles there too strong? Take them out” to “Now he looks too soft, put them back in!”. In the end the animation-model of the head had 23,088 polygons in the Maya viewport. The rendered model was composed of 369,408 polygons but was lacking displacement mapping of very fine details such as the pores, which got hand-sculpted, in the model as well. So in the end, with displacement mapping the model had 5,917,916 polygons.¹

And the work on the model was everything but over: For the animation-blendshapes Digital Domain used a FACS-based approach for which they needed to capture Jeff Bridges as he flexed single facial muscles as good as possible independently in front of a scanner. This performance got recorded live with a Mova Contour scanner that offered a realtime scan of up to 60 fps, although 30 fps were sufficient. The realtime scan was not as coarse as a LightStage scan, naturally, still the high-contrast make-up on the actor’s face provided the scanner with roughly 700 tracking points that were translated to vertices to drive the motion on the geometry, although lips and eyes were missing. The data was good enough to select on a per-frame basis the best “take” of single muscles that could be incorporated into the blendshape models. A talented modeler also took care of the transitions to areas the scans didn’t cover, such as ears when moving brows. In the end the animation-rig consisted of more than 200 blendshapes two modelers worked on for half a year. And almost as sophisticated as the rig was the GUI for the animators. “The rig is really well done, so you can’t break the model, no matter what blendshapes you mix and put on top of each other”. Still, that process was never finished and constantly new shapes needed to be added over the course of the production.

With the expressions now possible the young-Jeff head was posed to match various expressions of the photo and video-references of Jeff Bridges to see whether they would hold up. Only if it worked there, it was feasible to work on shots of the movie.

Approaching a shot

On the soundstage, Jeff Bridges’ facial performance was recorded with four cameras on mounted on a helmet that recorded his face with tracking markers from four different angles. Occasionally a witness cam filmed his performance from a fifth perspective, “although 90% of the time there was no room or time for a witness cam. When I suggested that, you know, even filming Jeff with a small handy-cam or cell-phone camera would suffice, the crew just shrugged that they couldn’t do that because they weren’t in the [cinematographer's] union. So we had to work with the head-cam footage most of the time,” Matthias recalled.

Instead of moving the skin according to the tracking markers on Jeff’s face, Digital Domain used a different approach: “Our solver tries to use the existing blendshapes to match the points as close as possible, so the animators can tweak, add and adjust the generated blendshapes,” although movement on eyes and lips is not recorded too well, so every shot is a lot of manual labor of a talented artist. Further, “there is no solver in the world that can correctly figure out what’s skin-motion and what’s bone-motion.”

The eyes proved to be a critical factor also in this production. The blendshape-controls around the eyes had to be refined and worked on by the two modelers with special care. Even simple blinking needed a proper deformation in the face. The eye-lines always came from John Reardon, Jeff Bridges’ body double on set, since it was John interacting with other actors and the environment.

John, the body double, would perform what he observed from the take with Jeff to reenact to the best of his abilities. His movements would get motion tracked so that the digital head could be popped on. Once the animation was done on the head, a preview was rendered with sub-surface-scattering so it could be evaluated how it would hold up with light and shadows; then with the hair simulation and if it looked right there, it would get rendered for the final shot, composited and color-graded.

But the animation was not just fixing what the solver couldn’t do, there was much more to it: For example the many “Jeffisms”, as Matthias called it, subtle expressions or looks, that the body double missed to reenact. “The problem is, that you can’t change the head motion much, since body and head are a unit and breaking their motions apart would look just wrong. This really opens your eyes on how much verbal communication happens through body language.” Still, sometimes the animators needed to add a little nod on a specific word in the head-animation or twist it differently.

Then there was the problem of a swimming or floating head. “We sent it back to tracking and what we got back still looked floating. We looked at the footage again and whereas Jeff was speaking his lines rather serious, his body-double played it much more casual and relaxed and his head just wobbled.” So the animators let the digital face smile a little more so that the lightness of John would be carried over. So sometimes you had to adapt the head to the body movement. “I can’t remember how often shot like this one”, he pointed at the bobble-head-shot looping on the screen behind him, “were sent to tracking back and forth and came back almost the same because the track wasn’t the problem.” In the lower left corner the version-number indicated that we were shown version 47 of it.

Occasionally the production team even rendered the head from the positions of the four motion-capture cameras with the same distortion they produced, “just to prove a point, since everything looks different when lit, moreover, Jeff’s lips look really big and different than from the shot perspective. This way the decision-makers could compare the original and the animated emotion one-to-one.”

Headworkers

Not that often, but still, the team encoutered the opposite problem, that there was too much movement in the face, or some “Jeffisms” were happening that a young guy just won’t do so that needed to be taken out. Something that did happen a bit more often was edited reference footage, so that Clu would deliver his line angrily, only to laugh in the end, such as with the line “Where are you now? Ha-ha!”. In those situations the animators had to manually blend the different expressions together as plausible and natural as they could.

During the animation an HDR image of the set was used for a quick lighting reference, but it also happened that the lighting changed drastically once the head was positioned in the scene. In one instance the head was not lit from above but from below. “A smile lit from above doesn’t need to be that big to have the same impression than when the face get lit from below like it was in this case.” Oftentimes it was not easy to tell whether the animation of the face was dead-on, so an intermediate rendering of a Lambert-shaded head with displacement was done, where the skin-details where much better to discern than with sub-surface scattering.

Lastly the sound and timing of the expressions needed to be edited occasionally as well. On one shot Matthias demonstrated the problem in the plate with the body-double: The first part of the dialogue he wasn’t looking quite in the right direction, in the second part his face was obstructed when he delivered his line. Again, the right transitions needed to be animated to get the timing right.

And also there were shots where there was little animation in the face going on from the beginning, like in a reaction-shot of Clu just listening. It always seemed too static and the director wanted a maximum of still plausible movement in the face, despite the freeze-frame-like appeal of Jeff Bridge’s reference from the shoot. “Sometimes people just don’t move!” Matthias said, adding “Still, you can’t ever totally stop animating a face, especially if it should look photoreal. This would give it away instantly.”

“Sometimes… sometimes shots just clicked together from start to finish and went exactly as expected” Matthias concluded his lecture. There were not many questions and I tried to get to the “Raum Ulm” for the Adobe presentation as fast as possible. Unfortunately I still was trapped between two rather heavy guys and so I endet up as one among many waiting and pushing towards the room’s entrance. A few people squeezed themselves through the packed crowd from inside the room and the fmx hostess only let in as many people as had left. I was not among them but still waited in front of the door, eager to get in to Adobe’s D-SLR Workflow presentation. Then the presenter, Michael O’Neill, appeared from the back of the crows with a bowl of sweets. “Gee, you all wanna in?” — “Yeah!” the crowd replied. I have never seen so many Canon 5D’s and Rebel Ti’s in such close proximity. “Okay, then can we do another session during the lunch break? Yes?” The hostess shrugged. “Alright, then see you all in my break then!” and he slipped through the door. “Cool guy!” one girl uttered. And then the crowd slowly dispersed and I went across the Schloßplatz in the Holanka Bar, a bookstore-café to start this report on today’s lectures.

Upon leaving I had a little accident that resulted in a graze on my nasal bone that began emitting a small but steady current of blood. The fun part is that I wasn’t aware of it until half an hour later when people eyed me more suspiciously than usual.

Adobe-Schmoby

Adobe’s notebook,
originally uploaded by Phil Strahl.

I returned just in time to queue up once again in front of the Raum Ulm for an Adobe presentation titled Hollywood Visual Effects. Again masses of people were packed like sardines in a tin can and waited patiently in front of the door. But this time I picked a better place and was among the first that had been granted admittance by the hostess. As I walked past her I could feel the stirred crowd pushing into the room like a herd of kettle. I don’t know how but she managed to keep them back. Didn’t matter to me, I had a seat in the front row.

Michael O’Neill arrived after the Geisha VFX reel had finished and seemed a bit stressed but happy to see an eager crowd. A few minutes in his presentation he asked “Who has worked with Premiere?” Many hands were raised. “Who has worked with After Effects?” Almost all of them stayed up. “I love you!” Mike beamed enthusiastically. I bet if he would have asked “And how many have 100% legal versions of our products?” his smile would have vanished instantly.

What followed was the VFX reel of Shutter Island with rather basic comments on how to use masks and keying, and then he re-made a so-so crowd duplication in a shot of Gareth Edward’s Attila the Hun movie. A tremendous no-budget effort but with… how do I put this… obvious not-really-state-of-the-art VFX. So frankly I was a bit bored by the very basic VFX 101 which might have been totally new for some of the youngsters among the crowd. At least I got rewarded with some candy for making a smart-ass remark. Nobody likes smartasses like me, but what does Michael do? He encourages my behavior by feeding me candy! Still, I was hoping to learn something new. What really kinda blew me away was the VFX reel of Boardwalk Empire because as I saw those sequences I not once even considered them being so VFX laden when I saw them in the show.

And in that respect I was rewarded because the presentation turned out to be a clandestine marketing plug for bringing the Adobe CS 5.5 suite to our attention with some cool new features such as the “Refinement Plate” effect in After Effects (or something like that) which basically is the Roto-Brush as an effect. Further I really enjoyed seeing at least some of After Effects’ effects ready for 32-bit or at least 16-bit color depth.

Mike also presented the new Lens Blur which, according to his presentation, lets you set the focus-point anywhere in your live-action plate and have the rest blur physically correct. When it was Q&A-time I wanted to know how this worked, like After Effects triangulated the content and calculated a quick depth-pass. But, alas, Michael had a tendency to understand questions the wrong way (not only mine) and explained in detail to me how the lens correction profiles in Camera Raw worked. I knew this. He even boasted “We have made profiles for all lenses and cameras that there are!” I really forced myself not to talk back, because just a few days before I downloaded Adobe’s little lens profiling tool because they obviously missed quite a few lenses in their latest update. Still I wanted to know about the Lens Blur, so I pressed the issue: “Yeah, but that was not my question…” He cut me off and started explaining how a lens worked, what the aperture does and so on. Sure, he couldn’t know that I most certainly know better than him how that stuff worked, still I felt like he treated me like an idiot who had a Canon 5D set to full-auto. Frankly, I was pissed.

Still I remained at my seat although it was lunch break. Micheal had promised us the DSLR Workflow lecture and really stuck to his word, although it was obvious that he wanted to get through it as quickly as possible to enjoy at least 15 minutes of lunch time. Nothing really new here either, just a little Dynamic Linking here, a little Adobe Bride interaction and bouncing everything to Premiere or After Effects. I don’t know why so many people look down on Premiere because Michael demonstrated its USP extensively: The ability to work with native DSLR-footage (and even RED and Alexa raw footage!) in real-time without the hassles of converting everything to your intermediate codec as it is the case with Final Cut and Avid. Since Adobe got their hands on the RAW-color-specifications, even camera-raw for RED footage in Premiere and After Effects did not affect the speed and quality of the real-time playback thanks to their Mercury Playback Engine. That was the most impressive thing I’d seen in this year’s Adobe presentation. At least something.

VFX Politics

Jeff Okun presenting,
originally uploaded by Phil Strahl.

Today I had enough energy and motivation for trying something new and so I made my way across the street and into the other building for Jeffrey A. Okun’s²Politics of VFX. I was lucky and got a seat right in the front row so view and sound were excellent and allowed me to take some good photos of the speakers as well.

VES‘s executive director and FMX veteran Eric Roth introduced Jeff, chair of the VES, in very dear and complimenting words so the man got a big applause despite we, the audience, quite didn’t know why we applauded so enthusiastically. This blind and externally fueled enthusiasm should really give us to think. On the other hand, well, we didn’t applaud Hitler.

“Let’s talk about politics!” Jeff begun.

“I want everybody to raise their right hand and swear not to ruin my already rickety career because I am going to name names and expose people. Now do it and say ‘I swear’!” Yes, this was going to get juicy, I thought with a smile.

Jeff started a –god knows– overladen and ugly Power Point presentation. The first slide read “Visual Effects and Politics. Politics show up when you are: 1. Getting Hired” and I counted five different fonts, three different fade-ins and also three different colors. I was overwhelmed by this visual bravery. Jeff climbed the highest point of the stage, directly in front of the projection and lifted the veil on one myth: “You don’t get hired because of your skills. You get hired because people think you are an interesting person, like the way you are or because you have a good reputation. If you don’t really have a dynamic personality you won’t even get hired.” And Jeff went on telling anecdote after anecdote.

“How many of you are industry professionals? … Good. How many of you are students? … And how many of you don’t know why they’re here but thought this was a good way to spend some time?” I already liked Jeff, whose polarizing character and appearance could be compared to being the VFX industry’s Julian Assange.

“Everybody on a movie has an agenda. And creating the best result on screen is usually never makes it to the top-ten of their priorities. On top are promotions, family, friends, future plans, etc. And this can lead to the strangest and most wacky decisions during the making of a film”, like when the whole movie gets shot in a certain location just because the girlfriend of the director, who happens also to be the lead actress, wants to visit Malta. Or that, for example, the production designer hates the VFX team because it threatens his job and so he sabotages their efforts in as many ways as possible on set. Jeff also told how on the set of Last Samurai Tom Cruise (whose high-fives were vicious to the point that you would duck away to escape them) was ordered to be shot in front of a blue screen — in a navy blue suit. “This was not because they didn’t know any better. This was because of proving a point, that the VFX crew follows the orders of the director and not vice versa.” It sounded painful. “When you’re not Digital Domain or ILM you don’t have much to say on the set and are under constant pressure to just get what you need.”

“Cinefex always presents the same old making-of story over and over again: We met with the director, developed the look, made some tests, he loved it, we shot it and produced the VFX and everybody loved it… Raise your hands, who of you has ever been in such an ideal production where everything worked from start to finish?”. Silence. I thought back about past and also present productions I took part in which where tattered and torn by the interplay of individual agendas. And in the audience not a single hand was raised as far as I could tell.

The next set of slides with sickening word-by-word-transitions materialized on the projection. As we all waited for it to finally finish, Jeff added with a charming childish smile “I had so much fun doing that!” The slides essentially said that all politics come from a certain point of view and that “your brain sees what it is expecting, not what is there”, a statement underlined with photos of optical illusion graffiti like these in Toronto “by a group of people who just do this, they say, to mess with drunks”. That got a huge laugh. Jeff then continued showing blivets, impossible objects. “Trying to get the right perspective hurts your brain like watching a bad 3D-movie” he spoke, not missing to make a small remark about the stereo-conversion of Clash of the Titans. “A blivet is to perspective what reality is to your brain” was prominently placed across a slide in a stinging yellow color. “Can you really trust your brain?” Jeff asked rhetorically “No. That’s the point.”

“Some politics are good politics, some politics are bad politics and some politics are just crazy politics. These crazy politics get propagated by crazy people, and artists are oftentimes crazy people. And where do you have a bunch of artists together? In a movie production. So 90% of the politics you get are crazy politics.” Jeff deduced. Was it really that simple? Well, it made sense though. “The thing is, that politics affect the story. The Story… this is the altar before we bow.” Jeff paused for a moment. “And, by the way, the thing that killed 3D three times now is not bad technology, it’s bad stories. Now look at today: The big studios are dependent on flat stories about super heroes. Soon we will have super hero TV shows and commercials. The thing is, studios make movies for business, we do it because we love doing it.”

Jeff then talked extensively about the issues on Red Planet, how the story sucked, how the leading actor pissed everyone off and sabotaged the whole shoot so “that you were lucky to get two shots a day.” The VFX Producer quit, there was no VFX supervisor and Jeff only stayed on the set because he was promised work on the newest Harry Potter movie if Red Planet was thgouh. Things heated up further, so that the two remaining lead actors (the story had killed the others one by one before) filed restraining orders against each other and the only way to finish the movie was by shooting them one at a time and compositing them together. All that sounded dreadful! And this was just one of many productions that were going down the toilet during production. “You can tell how bad things are going when you look at the versions of a shot.” One had the version number of v1156.

“VFX is another word for not being able to say no”

The problem of all this is that the costs increase and the VFX department gets blamed for “getting crazy”. According to Jeff, the budgeting usually works like this: “Some 21-year-old one reads scripts all day at a studio and evaluates them, like The VFX on this movie will be $ 800,000! Then I read the script and say: That will cost you $ 25 million! Then the studio panics and asks us to cut costs. With some trickery we figure we can do it for $ 19 million which still is too much for the studio. Maybe you should order a re-write of the script? I suggest and then the rewritten version comes back with a new scene of an army marching though a city to the harbor and I say: Phew, with all that we are around $ 28 million and the studio will reply in lack of understanding: Why the heck is it now even more expensive?!”
On every film Jeff (and I for that matter) ever worked on, the final number of VFX shots went up, and so did the costs.

“What only matters is what ends up on the screen. Nobody will ask you whether everybody had a great time or you barely made it out alive.” If the movie is a success, so are you. If not, and you’re the VFX Supervisor or Producer, it’s your fault. “But always be aware of the agendas and politics of the people around you.” That was heavy. But hey, somebody needed to say it.

Pioneering for future legacies

After a few minutes’ break Eric Roth once again made the audience tingle with excitement as he was announcing another industry veteran, Bill Kroyer, who talked about the making of TRON from 1980 to 1982 in How Classical Animation boosted CG.

Bill Kroyer, then and now,
originally uploaded by Phil Strahl.

Bill started with a short history lesson and with what it was like when he begun at Walt Disney Animation and got trained under the tutelage of the Nine Old Men. In the 1970′s the animation studio was the same as it had been for decades, Disney was very reluctant (if not avoiding) to adopt new technology back in the day. To get this point across Bill threw up a slide reading “The only technology in the studio not available on Snow White…,” the picture below showed an old electric pencil sharpener. You couldn’t start much lower on technology than that.

Around the end of the 1970′s Steven Lisberger became intrigued with video games and wanted to make a film in that aesthetic of glowing colored lines. Out of that idea he initially envisioned TRON as a fully 2d-animated feature in mentioned aesthetic, but the project grew bigger and so did the ambitions of him becoming writer and director creating TRON. He recruited a couple of animators and flirted with state-of-the-art technology in bringing the vision to life: by modeling the world and the vehicles in the three-dimensional computer-space to create “Simulated Images” as it was called back then. This was where the training as a Disney animator really set them on course because at Disney “you always think spatially, you create the illusion of depth,” Bill explained as he showed a character-board of Jiminy Cricket, drawn from any angle with the underlying three-dimensional primitives sketched in. This board was over 70 years old!

As for the look they got design legend Syd Mead work on the “evil” architecture and vehicles (such was the famous light cycles) and French comic artist Jean Giraud, better known as Moebius, to work on the “good” designs like the Solar Sailer. Moebius also was one of the first artists to rely in his storyboards only on outlines and different shades of gray markers. When the storyboards were finished and the few artists on the project knew exactly what they wanted it to look like, the hard part begun. For that matter each story board panel had two numbers on the bottom, the first one indicating shot order, the second one defining the exact length in frames.

“A lot of graph paper”

The Light Cycle,
originally uploaded by Phil Strahl.

There was no unified software for creating computer graphics at the time, there weren’t even unified computers, “the companies even built their own computers.” Four of these companies were tasked to recreate the designs of Syd and Moebius in 3D space with combinations of primitives, in the case of the light cycle it was MAGI/Synthavision in New York, a company that was founded in 1966 by Dr. Philip Mittelman, “the guy who also invented the MRI,” Bill added.

“At that time there was software for creating and describing stuff in 3d, displaying stuff in 3d, even lighting stuff in 3d but nobody ever thought of software for animating stuff in 3d” Bill explained. So “animating” became a completely different endeavor than in traditional animation:

322,200 values,
originally uploaded by Phil Strahl.

First, the animators needed to specify how the models would move in the digital world. For example, what the turn radius of a light cycle was, how it would tilt according to what curve angle and all that — not approximately but exactly, in values and degrees. The same route needed to be taken for the scene geometry: Everything hat to be drawn and outlined in the right proportions with numbers and curves on graph-paper; yet it didn’t end there: Animations of vehicles and the camera needed to be put into numbers and the classical exposure sheet now had 7 columns for each object: XYZ position, XYZ rotation (yaw, pitch & roll) and frame number — for each frame³.

“That means 144 values for one second of animation” Bill’s slide read. The next continued with “TRON had 15 minutes of computer-generated animation. That’s 900 seconds of footage. With an average of two vehicles and a moving camera in each shot, the animation required 322,200 hand-recorded values!” Heavy.

But Wait!,
originally uploaded by Phil Strahl.

“But wait!” Bill continued like a TV salesman, “there was one computer on the Disney lot in 1980.” It was the computer that was controlling the intricate moving panes which were used for pans and other intricate effects in Disney animation movies. And this computer could even calculate and interpolate soft Bezièr-splines, something your average curve editor does without bragging nowadays. So Bill would sneak into the animation department with the computer during their down-times (usually at night) and enter the start and end values. With the tap of a button the machine would compute and display the inbetweens. “But it didn’t have a printer, so you needed to record the values per hand,” Bill reminisced with a smile. And the operators at MAGI took these exposure sheets with manually recorded values and their people, Chris Wedge among them, typed them up for use in their software. That’s how early computer animation was done.

Bill talks about the Chromatix 9000,
originally uploaded by Phil Strahl.

“The first time we would see any objects in motion was in 70mm projected on the stage,” another slide emphasized. Because creating, calculating, positioning, rendering and outputting the animation was such an enormous feat there were no second takes, everything needed to be dead-on the first time. “Occasionally you got the chance to tweak a bit. So we had to know and envision each shot exactly and completely before getting to work, something that has changed with increased computation power over the decades” Bill subtly nudged the audience verbally towards the “think before you animate!” concept that gets so often left behind.

Luckily for the small team, new technology was available to them during the production in the shape of the Chromatix 9000 computer with a big screen, that could calculate and display shots in a rough wireframe, although not in motion. The transfer from 3d-workstation to the Chromatix was in fact via a phone modem and one screen refresh would take about ten minutes. “People in the studio would just drop by to watch that, shouting with glee how amazing it was.”

Optical Compositing

For some it audience the cumbersome post-production process of TRON was new and even for those who knew, the recognition was palpable: For the crisp and digital look of the movie, the soundstage needed to be filmed with a large depth of field which, naturally, required a big amount of light, even more so for 70mm footage⁴. Each frame of the black-and-white footage then got transferred multiple times to 11″ by 17″ film and boards on which various portions got masked out manually in the end by Taiwanese artists with paintbrushes and ink. These single masks then were then used for various different exposures that got optically composed back together which meant an insane amount of film-transparencies, masks and compositing-layers. Not even the color glow was out-of-the-box, instead it required thousands of wedges to get the right amount of glow intensity for each character in each shot.

And then there was the problem of flickering blacks, its origin in the pipeline the team could not pin down for a while until the varying black-densities of the Kodalith lith-film could be identified as the source, as Bill explains in the video below.

“You should get the BluRay of the old TRON it has been restored beautifully,” Bill plugged the movie, adding with a serious expression “but it if hadn’t been preserved on film, it would have been gone.” Bill was talking in the last minutes of his presentation about what he coined Digital Nitrate.

Bill during Q&A,
originally uploaded by Phil Strahl.

I was sitting in the first row and took a photo of that slide as Bill looked at me and pointed at my D-SLR. “Did you know that photographers at the Academy Awards carry two cameras, one being digital and the other analog? Because currently we have no way of reliably preserving anything digital⁶.” The last words also appeared on his final slide, erupted into Keynote-fire and vanished to ashes.

Instead of a grim ending like this one, Bill hurried to a more uplifting finish, mainly directed at the students among the audience, many of them eager for a job as animator — the sooner, the better. “People hire always the artist and not the technician,” Bill told them, “Visualize and communicate, independent of your software to become a better artist. Skill in a classical sense transfers to new technologies, so no matter what tools you use, stay focused on honing your skills.”

Applause. Then Q&A followed, some of it in the video just below. And a fun fact: Rendering all the CG-sequences in TRON together took less rendering time than a single frame of TRON Legacy. Amused chuckles in the audience.

Bye-bye

Unfortunately that was day two for me. I didn’t stay for the VFX of “The King’s Speech” and had by god no intentions for another round of Adobe’s presentations. In fact, my head was as tilt as any good pinball machine after a hearty bump so I left with the would on my nose for a bite to eat, a dose of caffeine and a smooth ride to the hotel.

The first row in many rooms is now reserved for lecturers so I had to abandon one of my favorite seating places of past years. Further the exit of the main auditorium has been moved to its side, probably because people pushing their way out and people pushing their way in made the access to popular lectures a pain — literally. Further, the FMX had matured and grown and hence offered even more parallel lectures in the opposing building.

Shortly before ten I had my ticket, had set off my first tweet, got my welcome bag, had my ballpen at hand and took a seat in the second row: I was ready for this year’s conference! Since I was a bit early I came to see the last of the screened animations of the Filmakademie Baden-Württemberg, a satirical take on the juicy topic of Austrofashism in the 1930′s.

Going Mobile

Neil Trevett,
originally uploaded by Phil Strahl.

Then they rolled this year’s FMX trailer, in my opinion a fresh idea after last year’s rather featureless animation. Then the first speaker of the event got introduced, Neil Trevett from nVidia, incidentally nVidia being the event’s main sponsor. I was preparing myself for having some marketing mumbo-jumbo dropped onto me but it wasn’t that bad after all in his lecture Movie Making and More, All in the Palm of Your Hand.

First he lined out how proud nVidia is of their Quadro series, CUDA and that all of this year’s Oscar nominated VFX films used nVidia technology somewhere in the course of the production. Big whoop — Adobe can claim the same thing probably.

Then followed some more nVidia-green Power Point slides depicting how the market of the “classic”, stationary desktop computer was more or less saturated today, even the growth for notebooks and netbooks started to level out, yet the market for mobile devices such as smartphones and tablet-computers exploded exponentially, with the Android operating system already ahead of Apple’s iOS. And since nVidia wants to get a big slice of that cake as well they started development of the Tegra series, high-performance graphics processors with as little power consumption as possible.

But that’s not enough: The old x86 chip architecture is also in decline since tablets and smartphones don’t come with a constant connection to a power grid, so the old ARM architecture will become a big thing of the near future. For nVidia this is Project Denver, ARM with integrated GPUs, while Microsoft’s Windows 8 will also support this kind of hardware. nVidia also works together with Adobe to make the execution of Flash and AIR applications more economic on mobile devices. “Apart from playing back video, the main task of Flash seems to be blending and filtering. And we let the GPU do this.”

nVidia seems to be pretty tight with Motorola as well, because their cutting-edge processors chips are and will be inside cutting-edge Motorola hardware as a chart showed. And I realized that I will carry a laptop with me for a couple of years longer, because what I want and need to do on the go can’t be feasibly done on a tablet anytime soon. But for the average person who wants to listen to music, surf the web, read eBooks and write emails a tablet with a full-sized keyboard/dockingstation/battery-combo (like the ASUS Transformer) might be just what they need. USB-input-device support anyone? Ice Cream Sandwich?

In the very near future mobile devices will have quad-core processors hence multitasking should become as common as necessary, further the portable devices will provide autostereoscopic displays, such as the Nintendo 3DS. Last but not least the resolution with increase drastically to an estimated 2560 by 1600 pixels (nVidia boasts it “Extreme HD”) which effectively results in a resolution of 300 dpi on a 10-inch tablet — print quality.

But what about power consumption? “As paradox as it may sound but more cores save you more power: The excessive cores only turn on when an application needs them” Neil explained.

Gee thanks, Neil, that’s pretty interesting, consumer-wise, but what about our industry? “Tablets in the movie industry can be and currently are used in three different ways,” he continued. One possibility is to port software to mobile devices. I snorted mentally: Anybody who tried the incredibly limited version of Photoshop for Android knows that a smartphone is still way too underpowered to get real work done. “By 2014 mobile devices will have 100 times the performance of today.” So will CUDA also work on phones? “No, CUDA’s power consumption is one of the biggest stepping stones in that respect” Neil shrugged. But that didn’t matter to me, because “the next generation of tablets and smartphones will be aimed at the demands of artists and the creative, like by additionally offering stylus operability and pressure sensitivity. The first devices will probably be released this year.” I dumbly smiled at the poor person sitting to my right. I am so buying into this!

The next useful integration of tablets was wireless tethering with desktop applications, because “this ‘new’ way of interaction with a user interface that is so much more intuitive than having to nudge a mouse pointer across the screen.” Adobe released for developers the Adobe Photoshop Touch SKD which opens developers the door to think of clever ways to integrate a tethered device. For example, it could serve as a color palette where swiping blends colors together. The resulting colors then can be returned to the host application. EDIT: Adobe recently me an announcement via eMail to update Photoshop and get complement software from the AppStore. Pity I roll with Android.

The third possibility is using tablets as cloud-clients. nVidia has already a Flash-based technology for combining collaboration with Tegra-powered tablets called “Studiopass”: It lets you upload, stream, annotate and comment in real time on video files with others. It is also possible to use the tablet as intuitive virtual viewfinder that’s connected to a render farm that returns within a few seconds the rendered image to the device. And, surprise, Studiopass is built on Flash.

Just when Neil got glowing eyes and popped up a slide in purest nVidia’s corporat0- design reading “Super” Computing, the time was up. Close call!

Baking Light

François takes a pic,
originally uploaded by Phil Strahl.

I was too lazy to cross the street for Approaching a CG Production so I stayed in the König-Karl-Halle for Bakery Relight by Thomas Driemeyer, Emmanuel Turquin and François Belot from The Bakery who talked about their company and their software, Relight. I was a bit distracted by the amount of people swiping away on their iPhones, Joseph Olin among them, while François outlined the history of the company in southern France. I only got to hear the last few words, “Sand, Beaches and Girls!”, although what I understood as “Beaches” could have been some other word. Or maybe it just was the thick French accent and the tendency to pronounce some words French, or switch to French vocabularies entirely.

Relight is essentially a tool for lighting and shading shots and to develop looks, realistic as well as artistic. “This tool was designed by artists and not by engineers” they noted and indeed: As Emmanuel showed a quick overview of the interface it seemed quite straightforward, a bit like Katana. “You don’t need to make a break every time you want to render a change on your way to the final shot, you always work with the final and that really fast.” Even little comfort features are implemented, like solo-ing certain light sources.

It plugs directly into Maya, 3dsmax and XSI, it comes with its own hair and fur system that is created at render-time by its production renderer that produces really good and clean images. The renderer itself is a Reyes-like rendering algorithm, iterative and optimized for fast feedback in production quality. But this need for heavy-duty-caching takes its toll on memory consumption I guess.

Once the scene’s visible geometry is cached by renderer (many millions of polygons take a couple of minutes) Relight unfolds its performance. Lighting and rendering a forest-scene with hundreds of trees and atmospherics only took a minute on an average notebook computer — impressive. And once the geometry and the first lighting pass is cached, the renderer gets even faster with every change. It keeps track of what has been touched or adjusted and only updates the dependencies, what had been affected by the change; and nothing more. “Depending on how many processors you use, the suite scales pretty well accordingly.” Even motion-blur and DOF (depth of field) are fast.

Another big thing of Relight are point clouds for quickly carrying out calculations that would bring a raytracer too its knees, memory wise. Point clouds (or disk-clouds to be more precise) are independent from the underlying geometry and make it possible, for example, to render ambient occlusion in fur stunningly fast. But ambient occlusion is only one application. Point clouds can be used for glossy reflections, environment light (from HDRIs, for example), area-lights or sub-surface-scattering.

Towards the end, François added that since Relight provides feedback so fast, it is a great tool for budding lighting artists and there are many collaborations with schools. Then he closed with a sentence in thick French accent that sounded to me like “Thank you very much for your invention!”. You’re welcome!

Arnold, the Tracer

Whereas The Bakery‘s Relight tries as much as Pixar’s RenderMan to cheat around raytracing, Sony Pictures Imageworks has a completely opposite approach with their proprietary renderer Arnold. The lecture by Marcos Fajandro and Larry Gritz Path Tracing and Unbiased Rendering started off by comparing rasterizer/Reyes renderers and ray-tracing.

Of course, Reyes depends on shadow maps and offers no ray-tracing but can be insanely fast, whereas a pure ray-tracer craps out completely with too complex geometry (see the issues with Davy Jones as discussed on the FMX/08) and/or/like hair.

So what do you do? Integrate some limited raytracing into a Reyes renderer or hack rasterizing & (deep)shadow functionality into a raytracer? Both options aren’t exactly irresistible. So the folks at the Spanish company Solid Angle, I haven’t heard of ever before, developed Arnold.

The first feature film rendered with Arnold was the, in my opinion terrible, Monster House and now since a couple of years it has become Sony Pictures Imagework’s only renderer.

Arnold is based physically on the Monte-Carlo methods, it uses no storage for lighting and global illumination information, has only one quality knob to tweak on (namely the sampling) and outputs the final in a single pass. Yes, Arnold is a ray-tracer par excellence and traces millions of rays. “And when you trace so many rays you need to make the rays really fast”. Further, Arnold sports networked, programmable shaders, subdivision surfaces and yet can handle “hundreds of millions of triangles and hair splines” and virtually “hundreds of gigabytes of texture-maps”. Impressive!

Since everything (everything!) is ray-traced, the final image is optically seamless. There’s no need for say, a shadow pass or motion vectors. My worst fear of such a renderer is that it probably takes ages just to get a feedback of lighting changes, but that is not the case: An example video showed a car in Maya in the viewport whose camera was turned around it. As soon as the operator let it go, Arnold kicked it and its interactive mode rendered the selected region in big blocks, then smaller blocks and even smaller blocks until you could make out the raytraced details of car paint and reflections. This refinement process took not more that two seconds.

“In the end what matters is not how long you wait for your beauty-rendering but how long you wait for feedback of each iteration in the (re-)lighting of a shot. Since an artists costs you approximately $ 40 an hour, this way of saving time saves you a big amount of money and the artist downtime between iterations” Marcos explained. I heard several mental kaching-sounds in the audience.

Making raytraced motion-blur efficient is also important for the final rendering. A fully lit and shaded scene with production-quality deformation motion-blur rendered only 15% longer than without. Arnold even is proficient when it comes to volume renderings.

“The joy of Arnold is, that everything just clicks together seamlessly when you raytrace everything in your image. If you use different renderes for different tasks, even simple things like shadows can be a problem, like shadows in volumes. With Arnold there’s no catch — you get the whole package in one go with ambient occlusion, motion blur, soft shadows on motion blur, caustics, etc. ” Marcos summed up.

And just as joyful is instancing of geometry that gets loaded into the memory once for whatever number of instances. One shot of Alice in Wonderland had 61 million triangles and could be rendered in a single thread’s 15 hrs which boils down to an hour or less in a farm.

“Still there is lots of room for future improvements” Marcos concluded before Larry took over. Larry talked in more detail about Sony’s way of VFX and relighting using the upcoming Smurfs movie as an example.

For every shot with CG-integration, they shot a HDRI of the set with an incredibly expensive Spheron camera. In fact, not just one panorama, but two in different heights so with the set survey data it is possible to calculate and to recreate the scene geometry via triangulation roughly in CG. Then, for quality reasons, the light sources and instruments in the panorama get replaced by Arnold lights and erased from the HDRI panorama via Katana. The actual plate from the camera is projected onto the low-poly scene geometry, then the HDRI panoramas get also projected into the scene. “It’s kinda the same what was done for Benjamin Button” Larry added. Now the CG-characters are imported into the scene and get their lighting and bounces from the surfaces that the plate and panorama was projected onto.

As mentioned above, Arnold has an open shading language. “Traditionally, shaders are black boxes to the renderer that just return some values to it; their units are sloppy oftentimes and their underlying C/C++ code can also crash the renderer altogether. With an open shading language, we don’t have these issues.” Instead of returning color values to the renderer, the Arnold shaders compute so called “closures”, descriptions of how the surface will react to light, and pass these numbers on to the renderer, which can decide what to do with them. “This is also 20% faster!” Larry added happily.

What working with these “closures” exactly meant and what it had to do with ray-budgeting and Multi-Importance-Sampling (MIS) for the Monte-Carlo raytracing I did not fully grasp but Larry had some links (and his email address) on the subject, in case anyone was interested. I reproduce them here for the same purpose:

• http://bit.ly/s10shaders
• http://opensource.imageworks.com
• lg AT imageworks.com

Some features of Arnold like sub-surface-scattering are performed with point clouds on-the-fly, although the renderer still employs Monte-Carlo methods when point clouds won’t suffice. Since Arnold is proprietary, Imageworks developed a plug-in for XSI themselves and a very basic one for Maya.

But why did Sony Pictures Imageworks settle for just Arnold a few years back? “The pass management grew so complex and bloated, it was hacks upon hacks, that overwhelmed the mental capacity of the TDs. Now it’s back to just having a single pass. Lighting got so much faster and in the end. Lighting and rendering a single pass still is faster than rendering lots of different passes and trying to get them working together.” Larry explained, and Marcos added: “When you reduce time somewhere in the process, however, somewhere somebody uses the new freedom to add more complexity.”

Currently Arnold uses only the CPU despite the trend of making everything CUDA- and hence GPU-compatible. “Porting would be complex and time consuming and in the end wouldn’t speed up things significantly, so we spend the efforts to optimize the CPU code instead. Further, the terabytes of textures would need to be streamed to the GPUs as well…” Larry summed up the bottleneck situation.

“We still have a couple of minutes left so I’m gonna show you the Green Lantern trailer. Everything was rendered with Arnold.”

Fun fact: The plate at the end of the trailer with the title and release date also had a rather small line reading “Also playing in 2D theaters.” Like it or not: Stereo has finally arrived and it is here to stay. Deal with it!

And here’s a LaTeX-set presentation on Importance Sampling for Monte Carlo Ray Tracing from 2006 with lots of pretty pictures (and some equations).

Coffee!

Haus der Wirtschaft,
originally uploaded by Phil Strahl.

The few hours I was asleep took their toll and I was wasted for the break. I schlepped my bag and camera to a long queue in front of Starbucks where I got me a motivational Caramel Macchiato so I had the energy to look for food. I found an Asian noodles stand in a subway station and made my way back to the Haus der Wirtschaft, munching on fresh cooked vegetables. The remaining ten minutes I had an espresso in the showroom of the FMX. When I felt ready for some more lectures and headed into the König-Karl-Halle and was surprised to see all the good seats taken. So I settled for a suboptimal seat next to a German student who was constantly eating or drinking something and merrily ignoring the ban on recording devices. Funny thing though was that he thought I didn’t speak German and I was in no mood to shatter his belief. So I sat there with Ophelia, my notebook, in her bag on my lap and waited eagerly for the next lecture to take place.

Render de Janeiro

Building, Lighting & Rendering “Rio” by Blue Sky’s Andrew Beddini was up next. Andrew somehow reminded me of a friendly Ben Stiller character and his engaging lecture was a pleasure to listen to without once the need to close my eyes for just a couple of hours. Or maybe it was because I was so close sitting to the pumped-up loudspeakers that I feared my ears would pop as they rolled the FMX-trailer.

Blue Sky, founded in 1987, is a veteran of the industry, they even took part in creating CG sequences in Disney’s original TRON. The next milestone was the animated short Bunny in 1998 I remember seeing at the Ars Electronica that time. I even remember that it was the first animation to employ the nowadays obscurely named Radiosity.

People really like to talk about their renderers I realized, as Andrew started summarizing the features of their proprietary renderer with the featureless name CGI Studio, or just Studio. On a rendering from 1993 Andy demonstrated the features their raytracer could produce back then which they still use today. Some of its features used for Rio were secondary rays, true radiosity, raytraced soft shadows, tesselation of beziér patches, a procedural shading pipeline and the possibility to use procedures for much more than just shaders. Rio is a feature of 1800 shots, and 80% of it had to be done in just 5 months so the focus was on creating a pipeline that was fast and efficient.

Character lighting was one of the main and foremost concerns of the production. And “good lighting needs good development.” The character of Susan with fair skin, glasses and big eyes was the first testing ground for plausible shading and lighting. First they applied the sub-surface-scattering-shader to her skin and realized, oh boy, that it looked not really convincing. So the first thing was to implement a density adjustment into it, then light transmittance through the skin, radiosity and a subtle but necessary secondary transmittance model that made the skin softer still.

Eyes behind glasses were another challenge because “eyes are critical for emotion. You really want to get the eyes right first. If they are off, if the audience doesn’t buy it, then all the other efforts are in vain, so get the eyes right!”. First they raytraced the eyes and the glasses but the physically realistic look did not work with the stylized character design. For example, the shadowing of the glasses under the rim was just too dark with the realistic refractive index of 1.5, only 1.1 was just about right. Can you do this? Should you do this? Andrew made it heard: “Break reality to make it work for you!” What was behind the glasses was rendered separately with a plate of the background to yield realistic refractions of it, as for the reflections, the reverse angle shot was used ever so subtly. But still the eyes didn’t look alive, so Blue Sky went the whole nine yards and rendered for the eyes a UV pass, a reflection pass, an object pass so the highlight could be adjusted in Nuke accordingly. This finally gave the eyes that certain something that was missing and was tweakable in every shot.

The set creation of the favelas was a difficult task as well. After the first color studies the assets were created with low complexity in a modular fashion and could be combined to a very organic whole according to a plan of the set’s final layout. “If you want to sell a set to your director render it in gray with just the ambient occlusion and he’ll love it!” Andrew joked. “There were absolutely no texture maps involved, everything was done procedurally with using the world space coordinates. Also we don’t need a level-of-detail (LOD) system.”

With just one building block, the lighting department instanced the geometry for lighting and mood tests of what the location might look in broad daylight, in the afternoon, at night and with atmospheric effects.

Raytracing and motion-blur — another dreaded combination of mine. But to tackle that problem Blue Sky employed a lot of tricks that made their lives easier. One of those was modulating the render-resolution accordingly via script: “At some point you can’t tell whether a 2k image was blurred or a 1k image was blurred, so if the renderer dials down the resolution you effectively are four times faster.” Another trick was using Nuke’s excellent vector blur: The camera and a Z-Depth channel from Maya got imported into Nuke and was blurred there in post.

A slide depicting the favela at night appeard on the screen. “In this particular scene we had about ten thousand light sources” Andy spoke and paused for emphasis. Ten thousand light sources with raytraced shadows?! “The thing is,” he continued “we use the same pool for all shadows, they get calculated at the same time. So if we’re having one light source or ten thousand only adds 10 to 15 % to the overall render time.”

Since the protagonists in Rio are birds, the sky is an important part of the film as well. “About 30% of the picture is sky!”. So how do you get a beautiful art-directed sky? The traditional answer is to have talented matte-painters. “The problem is, that we had four to five matte painters but hundreds of skies to paint. So this wasn’t feasible. Further the feature is in stereo, so…” he switched to the next slide depicting various clouds on a black background “…we had our art director paint a style guide for clouds we then built in 3D with volumes. We rendered each cloud with surface normals from 38 different camera tilts so you had almost any angle. Then we imported those onto planes in Nuke and populated the sky in the 3d-space there with the clouds.” The normals made it possible to vertex-light the cloud planes in Nuke. “This is really fast and you can churn out skies quickly. We did this for an entire sequence at once, not only for a single shot.”
But sometimes you need hero clouds, especially when they needed to have a volume and depth or some advanced lighting effects, like transmission on the edges of a key light. These clouds got individually modeled and rendered. The atmosphere-gradient with the sun was also done as a dome in Nuke and when you throw everything together — voilá — there’s your final sky! This sky then got rendered in Nuke and was used on a plane in Maya for the reflections on the water.

Generating a vista of Rio de Janeiro, especially a stylized version that still looks as photoreal as possible was the next challenge on Rio. The foundation for the environment was survey data of the topography that got artistically adjusted with some liberties. Still, it was built to scale so the renderer would deliver correct results of atmospheric effects, for example.

The vegetation was, like almost everything in Rio, procedural with implicit surfaces. But here the procedural approach posed some difficulties: it is hard getting a procedure to a single point. As for the shading, two world space procedural textures were created, one for the granite and one for the plants. A simple rule that depends on the steepness of the topology then blended between the two, so vertical walls would have no vegetation and soft slopes would be fully covered in trees. Finally, the city’s buildings were roughly modeled, but also were designed for procedurally driven variety: They could have differently spaced windows, doors, floor-segmentation, surfaces and rooftop structures.

What surprised me the most was the way the lighting was defined and carried out: It was script-driven and those scripts were simple text-files the production edited with NEdit in Perl and Python. “This is great because the files are small and portable, you can send them securely via email, and you don’t need to open Maya just to twist a light a bit to one side.” Blue Sky even has an interactive front-end called “Quick Render” to render models and light situations without waiting for Maya to finally launch. And there was more: “Light sources are not defined in the world-space but in camera-space,” instead of XYZ they are described by values for rotation, elevation and distance from the camera. Again, you don’t need Maya to adjust a light; creating a consistent rim-light can be done with a few lines in a text-editor easily.

Finally, even the atmospherics were rendered in favor of a Z-pass. “A Z-pass always has artifacts along the edges so you need to render it really big.” And once you have time for that and to add fog & haze in post, you can render physically much more plausible (and within the scene reflected) atmospherics.

A Hairy Subject

We only had a few minutes to let sink in what was heard before Mohit Kallianpur from Disney Animation Studios continued with his lecture of Untangling “Tangled” where he was the Look & Lighting supervisor and described the history of the look of the movie.

In 2007 the movie had a much darker, browner tone to its concept art until John Lasseter intervened and pointed the production in a more colorful and saturated direction. Still, that was relatively late in the pre-production stage and there was not that much time left to get the movie done. So Mohit got down to the root of art direction and formulated three principles: Stylized shapes, illustrative colors and believable textures.

Shapes

Then the research begun by watching and analyzing the shapes, colors and appeal of the old Disney classics Snowhite, Pinocchio and Cinderella. Especially the latter had a certain shape language of flowing curves visible in almost every shot, a graceful harmony. Moreover, a set of signature shapes of that movie was collected consisting of various bell-shapes, s-curves and leafs. Everything in the film would follow these shapes, even the canopy of the trees.

Based on a very impressionist and rough mood-painting that followed the shape-guide, the team produced a full CG version of it as reference. This made it obvious that the language of shapes worked well, but the painterly appeal of the surfaces was too stylized. So the world needed to have believable textures, not rely on impressionist suggestions of detail.

Architecture

The architecture of Tangled was influenced by European cities, Disneyland (!) and Pinocchio: Everything should be small, friendly and approachable. The buildings are not tall and they flare out in a curve, they appear even chunky and beefy with no sharp corners and a very organic and hand-built feel. And like this Mohit wanted the architecture on Tangled to look: old and used but not decrepit or dirty.

Color

But what does “illustrative color” now mean exactly? The production settled for a lush saturated palette and always a play of warm against cool: If the light is cool, then the shadows should be warm and vice versa.

Legally Blonde

Yet the most intricate and most critical task was to get the look of Rapunzel’s hair perfect. Much like the sky in Rio, the hair in Tangled was a character by itself. The first tests showed that the hair shaders the studio had developed so far didn’t do justice to blonde hair. At all. So research needed to be done and a hair model was found and her hair photographed in probably any appearance (even wet) and every possible lighting situation. Also commercials of hair care products served as a valuable reference “since they propagate the ideal on their packaging we wanted to recreate in the film.” Even a PhD student did extensive research on the subject and in the end a feasible shader was programmed:

First of all, hair comes in individual strands and strands that make up a volume of hair. Moreover, these strands have a top, a bottom and a body with an uneven surface, none of which is like the cylinder-representation of hair we all used to work with. So hair has a specular reflection, so far so good. And for black hair that’s usually all you see. Light hair also has a sub-specular portion that is a broader highlight in the color of the hair, whereas the specular highlight has the color of the light source. Then hair has a transmission value when lit from behind, and multiple scattering is important for light colored hair. Still that’s not enough because a hair rarely comes singly: There’s also a volume diffuse portion, that is backward scattering of the light and volume transmission, which scatters away from the light source through the volume.

Now if you add up those five components it looks good but not perfect, a certain richness is missing. This “richness” comes from the light bounced back onto the hair and ambient occlusion. Then, and only then, you are rewarded with beautiful blonde hair. But for eyebrows, fur and eyelashes you still need a different shader because the blonde shader needs a volume to work with after all.

Appeal

And another question: What constitutes appeal? What makes a person look appealing in a movie? How to light a character to look appealing? Mohit dug deep once again and looked through lots and lots of glamor-shots of actresses and actors of Hollywood’s golden age in search for unifying principles. And his answer is simple: “Cheat!”

The starlets on photos (and in movies) were always lit by a soft light, no matter what the rest of their environment looked like. Then there needed to be hue in the shadows, the dreaded “graying” of multiplying occlusion with diffuse passes makes characters look sickly. Since the eyes are the window to the soul, they needed special attention and always a specular reflection, no matter what. And as painters suggested, the cavities of mouth and nostrils should not go completely black but into a warm darkness. And unappealing colors (usually green bounces of leaves and grass) also make a character look weak and sick, so they had to cheat there as well. Finally a subtle bloom on the highlights never hurt anyone.

Look

On Tangled there were 19 look development artists that in the end produced over a thousand paintings of looks. And lighting was important to elevate the mood that was already there by narration, staging and framing. As Mohit showed some examples of lighting I could not help but remember the words of my mentor, Fraser Maclean as Mohit uttered them as well: “Put light where you need it, not where it realistically would be.” Indeed, the progression of shots he showed were lacking any continuity of the origin of the lights. “And here… well I don’t know where the light is supposed to be coming from, there is no window there” …but still it works perfectly in each shot. It went so far, that even the murals in the tower could be toned down or changed in opacity on a shot-by-shot basis. “Light shapes add to the drama.”

And so does saturation. In dramatic sequences the saturation got sucked out of the pictures, only to return almost fully later on.

Art-Directed Trees

Once you can art-direct everything, you have to art-direct everything. The R&D department provided a simple tool for Maya that allowed the modeling artist to draw a couple of curves and the plug-in would make a tree with branches out of it. Moreover, they also had a tool that would grow leaves into a pre-defined canopy-shape which was really fun to watch.
And since there are so many trees in a forest, the models were also switched to “brickmaps”, a RenderMan-term for low-res voxel-representations of a model, if far enough away and automated “stochastic pruning”, which meant that depending on the distance to the camera, not visible leaves would be automatically switched with low-poly models or removed entirely.

“In the end we generated too many trees and had to hide them in haze or atmosphere in post” Mohit admitted. That really was new to me, having accidentally too many in the final picture of something so complex such as trees!

Tangled was rendered with RenderMan (finally another Reyes renderer today, I was almost worried!), had 1380 shots and 55 lighting artists which, in sum, resulted in 9.01 million hours of a single lighting thread. “That’s more than 1028 years”. Ah, statistics. It’s like I tell you now that this blog post is already some 6000-odd words long.

Physically Based Shading

Ben Snow,
originally uploaded by Phil Strahl.

I used the short break to get back close to the front in row 2 and to empty my can of Starbucks “Doubleshot Espresso” to be all up an ready for Ben Snow’s and Christophe Héry’s Physically Based Shading at ILM¹.

Ben showed some definitions at first outlining GI, IBL and HDRI. “We’re not trying to reproduce reality,” Ben made clear, “but filmed reality” and described a little ILM’s history in their attempts to achieve this goal over the years.

The practice of trying to capture as much information from the set grew over the years in scope and professionalism as well. In the 1990′s they started filming and photographing light probes on set, 18% gray balls, to recreate the light later in the computer. But soon they realized that was not enough. Along came six photos in each direction for cube-maps and then the familiar chrome ball, a technique I never quite got to work for my own projects. Indeed, the chrome sphere’s reflections were rather low-res and you needed to paint out the photographer every time as well.
So in the “early days” ILM made heavy use of texture maps with painted in highlights and shadows, for shading they employed the Cook-Torrance specular model. Their light rigs were also pretty basic but could handle real world situations already rather well. Occasionally shadows grew really dark and some ways to cheat were e.g. spot lights or churning down the overall shadow opacity².

Then came Perl Harbor when they really needed to crank their depicted reality up a notch since ambient occlusion wasn’t enough. So they developed what they called Ambient Environment Lighting, which essentially is creating a pass for the ambient lighting via ambient occlusion: Rays are cast in a hemisphere around the surface normals, then a number of rays hitting other surfaces dictates the occlusion; and a pre-pass is done to calculate the average direction of the light. At least that’s what I copied from Ben’s slides. The film was a milestone nevertheless because Michael Bay, the director, was not able to tell the difference between CG and live-action-footage anymore.

What happened after Perl Harbor was boosting the quality consistently further. 8-bit images had served their purpose well (including in Perl Harbor!), but the need and time asked for floating point precision. Also, no mirrored balls would be photographed anymore in favor of photographed panoramas.

So a couple of years afterward came Iron Man and asked for the realistic depiction of, well, iron and metal that needed to match the practical suits on set. And a thing that had been troubling the folks at ILM was anisotropic highlights that appear on brushed metal.

Further, a new approach for taking HDRI panoramas was employed and meant taking a series of photographs from a tripod in all directions, although (as it appeared to me) not in a truly high dynamic range but only covering two two exposure brackets — too little from my experience but still it worked for them.

New Frontiers

And just as they thought they had mastered metal, along came Terminator: Salvation where it was impossible to cheat any longer and they had to move to a new paradigm for lighting and shading in RenderMan. ILM’s goal was to get a simpler, more intuitive and physically based system of lighting and rendering:

The quality of shading and lighting requred a BRDF-model that not only looked right but also acted physically correct in terms of energy conservation. In short this means, that the rougher a material is, the weaker is highlight gets, otherwise the energy (= light) reflected would be bigger than the light received — impossible³.

That also meant a normalized specular highlights with a more physically plausible specular falloff, so the further away a light source is, the weaker the specular highlight gets, depending on the roughness or the “Normalized Importance Falloff”: The intensity of the highlight falls off on rougher surfaces. For tight specular highlights like chrome or mirrors the light source has to get a long way away before dimming. The broader the speculars are, the more quickly they will dim. For this to work the light needs to have a physical size in the system, so no more point-lights or directional lights at ILM

“This was hard for everyone to adjust to and Christophe and I remember some very passionately fought holy wars” Ben remembered.

On the set of Terminator: Salvation the chrome spheres were back, but differently. Now they were moved and shot in motion so their reflection could be used on moving models. “We still don’t have a great way of capturing HDR moving images,” Ben continued in front of a turntable of a T-800 in front of a steelworks plate, “instead we shoot HDRIs with stable lighting and we also shoot our chrome spheres so we still get our FX, strobes, sparks, etc. And on top of that we applied some pyro elements shot on film and used them as reflections or area-lights in the scene.”

Iron Man 2

“We now had our tools that worked, they weren’t really mature but technically they were robust” Ben noted. For image based lighting ILM then used a graphical tool, the Environments Browser, to quickly define light sources within the image so they could be recreated with area lights. The match-move of a shot creates the environment and the HDRI panoramas get projected onto the geometry and you basically end up with a HDRI-mapped recreation of the set environment in 3d. That enabled the artists to render dynamic HDRIs from any position in the set to be used in image based lighting of the digital characters.

IBL via HDRI

But Image Based Lighting also poses some risks or at least things to watch out for. An important thing to be aware of is that IBL-lights are treated as infinitely away point-lights from the scene. A fact, that comes distractingly into play when CG-lights with a discrete position within the scene are meant to work alongside an IBL dome. Then it really takes an experienced photographer and visual effects team to produce good and usable HDRIs from the set which is essential for this approach. Last but not least the pipeline should support the the floating-point nature of HDRIs or else one might run the risk of losing dynamic range along the process when editing HDRIs.

On set ILM still photographs chrome and grey spheres but only as references. This only works properly if the film crew is in the habit as well and does not treat those shots lightly or worse, forgets about them. Since the VFX team later needs to match the lighting of the shot everybody liked on set, those references should be shot immediately after the director yelled “cut!”. “You wanna make sure that your spheres are as big as possible in the frame and you wanna make sure they are in the right spot. And you don’t wanna shadow or be reflected in the sphere” Ben reminded. Sometimes when the spheres are moved like an object though the scene, it is advisable to take some static sphere footage as well.

“On the set as soon as they say We’ve got it! then off you go and get the references.” These references, of course, should only be shot for shots that will have CG portions but if there is ever any doubt that a certain shot definitely will not require CG, one should still capture the references to be on the safe side.

“And be serious about it. If you don’t take this seriously nobody else on the crew will” Ben shared his experience with the audience and switched to his last presentation slide, titled “How we capture HDRIs” which I shall reproduce here:

• Canon 1Ds Mk3 with Sigma 8mm fisheye lens
• Nodal Ninja & Tripod
• Remote shutter trigger
• 0.6 ND (2-stop) filter (depending on how bright it is)
• 7 exposures, 3 stops apart
• Direct sun f/16, ISO 100, center exposure 1/32 sec

After a quick slide of acknowledgements Ben handed over to Christophe who would once again talk in a little more detail about the math-laden background behind the presented concepts.

Since my senior-high maths-teacher sucked out all the fun I ever had with mathematics, I wasn’t in the mood of trying to follow Christophe’s every word but I got the general idea:

Calculating reflections and IBL with the Monte Carlo approach requires a high amount of samples for a clean picture. Since the distribution of the rays is random, you end up calculating a lot of stuff you will not really see in your final rendering. So he came up with MIS, Multi Importance Sampling, that takes into account what the camera sees, that the light illuminates and identifies an area in the lighting-dome where rays have a probability to affect the final rendering, such bright portions in the HDRI used for IBL. With the same number of samples used for rendering, which are only weighted differently, in the end compose a much more pleasing result because you only calculate what you need.

To implement this into the renderer, the BRDFs and lights need to provide eval() and sample() methods, where eval() returns a color and a pdf for a given input direction and sample() that returns an array of directions and pdfs: For instance, in a dome IBL situation, these will be the vectors to the bright spots in the image.
I just copied this from a slide of Christophe’s presentation, so don’t ask me what it all means. The only thing I am pretty sure is that pdf in this context stands for probability distribution function and not for Adobe’s favorite way of storing their user guides.

ILM’s BRDF specification has to follow a number of principles too: The shaders now must be normalized e.g. energy conserving, they must “substitute” to ILM’s trusted but old Cook-Torrance model, they should be anisotropy aware and, as always, efficiently computed.

Their solution now is D-BRDF based on an yet unpublished paper by Michael Ashikhmin and Simon Premoze; the Beckmann distribution⁴ and no masking: the reciprocity term is simply 4.0 * V.H * max(L.N, V.N).

For the nerds among you I even noted the links Christophe’s presentation closed with:

• Eric Veach’s paper⁵
• Last year’s SIGGRAPH course
• D-BRDF

You’re welcome. Now where is my mind?

After this heavy information laden day I skipped nVidia’s panel discussion and went straight home to type up this blog post. who would have thought that it would take me effectively over two weeks to finish it?

Creating Games Back Then

So Jot is a game designer, isn’t that great? We had a little chat about how it all began and he was raving about an ancient program by Europress Software called Klik & Play. And boy, how well and dear do I remember that program! I got it as limited shareware with one of the first PC magazines with CD-ROMs, actually from one titled “PC Spiel mit CD-ROM” that literally translates to “PC-Game with CD-ROM” (I even blogged about it!). With this software you could create simple games across various screens, with your own animations, your own game mechanics and your own story. It was like Mario Paint on meth. To me as a 12-year old it was creative heaven.
The limitation of the demo was only in content, you didn’t have as many pre-animated sprites and backgrounds to choose from and your games were limited to only two or three screens¹. But the successive issues of PC-Spiel mit CD-ROM supplied me with games by other homebrew-creators: There was a real-time Worms clone, a game where you had to break out of prison and escape the police or a medieval fighting game. Best of all: I could open those games in K&P and, voilà, I dodged the screen count limitation of the shareware version. As a kid I obviously was pretty handy levering out that kinda stuff. I even went through the hassle to rework some of the original games with better graphics, sounds and animations. As a kid I was also an obvious smart-ass. But it was fun. Like Mario Paint on meth.

The successor of Klik & Play was the Games Factory and ultimately Multiumedia Fusion, the software that Jot is currently using to make his many games.

Pitiri or the importance of being obsessed

About two years ago Jot started working more or less alone on his awesome retro-styled jump ‘n’ run called Pitiri; although ‘retro’ in the sense of 1970′s hairdo and music, not so much ‘retro’ in the sense of chunky pixels in stunning three colors. No, Pitiri is different, yet so comfortably familiar.

When I visited Jot his room’s walls were plastered with designs of levels and enemies and the game-flow itself, hell, they still *are*! It took him many months and hundreds of hours to create the world and narrative of Pitiri. Apart from the intro, some sounds and graphic assets he did everything himself, occasionally having a fistful of friends as unpaid beta-testers. I was lucky and honored to be among them and gave poor Jot a hell of a time with my many pages of bug-reports for a simple single level. I’m a smartass, at least that’s what people keep telling me. Jot also composed and performed every musical piece in the game and, until I learned about that, I could swear I heard some old but good Neil Young song in the beginning.

The game still isn’t finished but what I have seen and experienced so far from Eli’s quest for his abducted brother I can assure you it’s gonna be great! I mean I talked with a robot, traveled in space and could turn myself into fire! Even at the same time, if I wanted to!

The requests

I am a big fan of independent artists, regardless of their medium. So I like to help out Jot occasionally with my ruthless selflessness … ahem. Okay, who am I kidding? I do it for the fame, the money and the girls²!

For example, Jot needed a “playful, computer-styled retro font” so I spent one or two afternoons with my sketchbook in my favorite café and designed him a font I titled Digital Disco despite the fact that there’s already a font by that name, as I learned much later after a quick Google search.

If you’re interested in my font, you can get it here as donation ware, which means you can have it for free but not clicking either the Flattr, the Kachingle or the PayPal Donate button makes Baby Jesus and Baby Moses cry. And nobody wants that, right?

But there was a new request: A couple of days ago my good friend Jot from ilikescifi.com told me about a new game of his in beta, called “Aratatatt” which will be available in the near future on the interwebz (I keep you up to date). It’s a jump and shoot in a post-apocalyptic world populated by evil robots. Since the game was all retro again (the synth-pop and new-wave soundtrack by none other than Jot himself) it featured a level boss. And Jot wanted 8-bit sound for it and asked me if I could supply him with a loopable chiptune that would last a good 40 seconds at least.

That’s what he told me via Steam when we were roaming our individual irradiated wasteland of the Mojave desert of Fallout New Vegas. As I was in a tough fight in Vault 3 with some fiends I got the guideline from Jot: “I’d love it to be just like a great Metroid-like showdown. I want 8-bit metal!”. I walked in the rearmost chamber of the vault with my companions Rex and Cass as I heard another message come in from Jot: “It’d be great in the next two days or so!” Only two days?! I took down Motor Runner with a skilled headshot and exited the game. I had work to do.

The tune

I switched on my Yamaha keyboard and launched the magnificent FamiTracker, a capable tracker that allows anybody interested and nerdy enough to make her or his own NES-compatible chiptunes with it, and hit some notes. It sounded terrible. I took some more time to tweak on my instruments, to come up with a rhythm and melody, hit the keys and — major suckage again. After an hour I was in grave despair and spent the following hour looking for kick and snare drum samples to use in the DPCM channel. Luckily end-boss-8bit-metal has a fairly straight-forward percussion. Really bland.

Then I tried coming up with a melody or at least a rough direction to the sampled drums, using the noise channel for hi-hats. Nada. I was incredibly frustrated until I decided to listen to some game soundtracks in my vast library. I hit gold when I dug up the soundtrack to Legend of Xanadu. The first part in my tune is largely based on the track JEMEUX MOUVAIS in the game, whereas the intro was inspired a little by XARKAS although my lacking proficiency in chiptuning really shows when you compare my tune to these two songs.

So finally I could start with the first part and it sounded really good and I got really far by my standards: 20 seconds. And absolutely no clue how to stretch it to 40 seconds. So I sat there, in front of the computer, the sun already up again. That’s when I went to bed. Luckily I was on a night shift the next day. So when I came back from work around 3 a.m. the next day I sat straight to my Mac and moonlighted color grading on some another project. Whenever I had graded enough shots for a decent render time, I switched to composing on the chiptune until the next batch of shots was ready to be colored. Despite what anybody’s common sense would imply, this back-and-forth of two totally different demands was highly effective and I managed to compose two patterns of nice breaks for the tune. This helped me out another 10 or so seconds but I still wasn’t very close to 40, and again, out of ideas.

That was when I really took the coward’s exit and after half the song I just transposed everything up one note and repeated what I had so far with some minor variations. I played it and took the time. Almost 50 seconds for the loop. I was free! And the sun had already risen again. But I wanted to be sure that it would sound good enough for Jot, so I copied the compiled song onto a CF-card, popped it into the PowerPak and turned on my TV. I wanted to know what it sounded like on a real NES.

It sounded holey, a lot of instruments weren’t playing and I was pissed. I went back to the tracker and tweaked around the omitted sounds in the hope of eradicating the problem. I don’t know what I did but the next time it worked. Like a charm. And it sounded really thick and familiar. Thanks to the system’s bias to let the song more or less completely clip and apply a rather stringent low-pass filter. But it was an analogue and warm clipping and the low-pass only made it less clicky.

As soon as I was done I sent it off to Jot and awaited his reaction. I was really hoping that I didn’t disappoint him too profoundly but a couple of hours I got his feedback: “shit phil it works so fucking well! you gonna love it. shit! thanks.”

Despite all the profanity it’s still the best reaction I ever got to a chiptune of my own. And since it is now officially Jot-approved I thought I might share it with you. You can listen to it here or even download the .NSF-file below. Enjoy! And you might want to click the Flattr or Kachingle button…

Seph Carissa – Showdown. 2010.
Download audio file (seph_carissa_-_showdown.mp3)
No player? Try this.

³
Download .NSF

Level Information:

• This level occupies 13 MB in your memory and has a time limit of 14:14 min.
• This podcast is EXPLICIT because I of my occasional pottymouthism.

If you like it then feel free to click the Flattr button on this site, follow the BleepCast on Twitter and/or drop me a comment. Thanks!

Level Information:

• This level occupies 51.5 MB in your memory and has a time limit of 56:27 min.
• This podcast is for once NOT EXPLICIT because I could keep my f**king mouth shut – yaaay!

• More information about the games Hubbard made music for, including cover art and personal experiences by the author at The Commodore Zone.
• An Interview with Rob Hubbard from 2001 by Neil Carr.
• A really comprehensive Interview with Rob Hubbard on c64.com
• Rob Hubbard’s unabridged speech at the “Assembly 2002” Demo Party in Finland, found in stormblast0891’s YouTube channel.
• Rob Hubbard’s Wikipedia page. Somebody should edit it with additional information!
• An interview from 1997 with Rob who was still at Electronic Arts back then.
• Not the Rob Hubbard you’re looking for.
• An awesome amount of remixes of Rob’s tunes at remix.kwed.org.
• Too lazy for browsing the High-Voltage SID Collection? Here are almost all of Rob Hubbard’s SID tunes.
• The starting point for your own research.

If you like it then feel free to click the Flattr button on this site, follow the BleepCast on Twitter and/or drop me a comment. Thanks!

Level Information:

• This level occupies 41.5 MB in your memory and has a time limit of 45:27 min.
• This podcast is EXPLICIT because occasionally I get quite profane…

• More information about the games Hubbard made music for, including cover art and personal experiences by the author at The Commodore Zone.
• An Interview with Rob Hubbard from 2001 by Neil Carr.
• A really comprehensive Interview with Rob Hubbard on c64.com
• Rob Hubbard’s unabridged speech at the “Assembly 2002” Demo Party in Finland, found in stormblast0891’s YouTube channel.
• Rob Hubbard’s Wikipedia page. Somebody should edit it with additional information!
• An interview from 1997 with Rob who was still at Electronic Arts back then.
• Not the Rob Hubbard you’re looking for.
• An awesome amount of remixes of Rob’s tunes at remix.kwed.org.
• Too lazy for browsing the High-Voltage SID Collection? Here are almost all of Rob Hubbard’s SID tunes.
• The starting point for your own research.

If you like it then feel free to click the Flattr button on this site, follow the BleepCast on Twitter and/or drop me a comment. Thanks!