From Imagination to Pixel: Behind the Scenes of The Book of Life
The Texas-based VFX studio, Reel FX, was founded in 1993, but it made its first entrance into the world of in-house animated features in November 2013. Its release was Free Birds, a quirky holiday tale of time traveling turkeys set to save Thanksgiving. It’s been less than a year since then, and the studio has already released its second in-house animated movie, The Book of Life this October. With just three weeks in, the film has shown some positive results both critically and financially, already grossing over $80 million worldwide at the time of this writing.
A large part of The Book of Life’s success is due to its unique visual style. The film breaks the visual mold that's being used to produce a good majority of animated movies today. Its storyline is also quite distinct in that it takes as it subject Hispanic culture in general and, in particular, one of Mexico’s most beloved traditional celebrations, dia de muertos (the Day of the Dead). The story is an ambitious retelling of the origins of the Day of the Dead and spans three completely distinct worlds. Each setting needed to have its own look and feel. More importantly, the film was a culmination of over a decade of planning by writer/director Jorge Gutierrez who already had definite ideas about how he wanted the story to be represented visually.
It was up to Reel FX’s talented, but small, team of CG artists, VFX specialists and supervisors to help bring Gutierrez’s vision to life. When the final images were rendered, the team had succeeded in incorporating a wide variety of visual styles, creating some innovative custom-built tools, and building an atmosphere based on close collaboration. We were given a first class tour of the studio's production by several of its supervising members: Paul Sullivan (Art Director), Augusto Schillaci (VFX Supervisor) and Glo Minaya (CG Supervisor). All three provide a unique perspective on understanding how the studio's process helped realize the director’s epic vision with only a fraction of the budget allotted to most animated features.
Conceptual drawing of "The Land of the Remembered" by Paul Sullivan. Twentieth Century Fox & Reel FX[/caption]
There was a general feeling among every member that bringing the director's vision to life was a top priority. Therefore, the team paid strict attention to what the art department delivered. One thing Gutierrez insisted upon was that a fidelity between the film's artwork and what was on the screen. "That was one thing Jorge really wanted. Sometimes you see the artwork for the movie, and it looks much nicer and more complex than how the movie ends up being. So, we set out from the beginning to make sure that didn't happen."
Coming up with the overall look for the film took some initial experimentation. By developing the look for a particular scene, the team could then use that look and approach it as a model for applying those characteristics to the rest of the film. One particular scene that was used for this purpose was when the character Manolo is resurrected from the dead after being bitten by snake. "That shot took us three-and-a-half months to do. With that shot we came out with a look dev of how to do all of the other shots. So, it took us longer in the beginning."
Park Guell designed by Antoni Gaudi. Photo by Bernard Gagnon[/caption]
Like many directors, Gutierrez also brought to the project many specific personal references for the film’s story and visuals. Schillaci explained some of these artistic allusions, many of which seem to make for some strange artistic bedfellows. "Jorge is very old school when it comes to the movies. He incorporated a lot of old references within it. For example, he wanted the Land of Remember to be kind of like the city in the film Metropolis. He wanted the city to feel infinite. Of course, in Metropolis, you end up having a ground, but he gave us the reference of Metropolis for how we should structure the city itself.”
Another aesthetic influence on the film's look included famous Italian director Sergio Leone and his "spaghetti westerns." Much of the color profile for the Book of Life's town of San Angel was intended as an homage to Leone's films. This included creating a sepia-toned pallet for the town scenes. Art Director Paul Sullivan gave us some insights into how this approach worked. "When they shot those [spaghetti westerns], they would shoot day-for-night and would put a night filter on top of it. So we kept that in mind with our coloring and lighting." Sullivan also explained that the film's frame narrative design (i.e. a story within a story) was inspired by The Princess Bride. "In the beginning of the Princess Bride there's a narrator who's reading the book to the kid, and we do the same thing in our movie.
Gutierrez's personal references for his vision of the film also extended to video games and other pop culture references. At one point the director wanted a particular scene featuring Joaquin and a rooftop battle with the Banditos to resemble the video game Street Fighter. "It's not like the animators came up with that idea. Some of that really was Jorge putting in a lot of different references. Like for example, Xibalba's castle...the one in the lava... was a reference to Snake Mountain from Masters of the Universe, the He-man 80's cartoon," recalls Schillaci.
Initial sketches of Manolo's arm. Twentieth Century Fox & Reel FX[/caption]
However, creating a doll-like, folk art look posed some problems for getting believable animations. Sullivan explains how one aspect of the character's arm structure presented issues. "Manolo's arms were extremely difficult to figure out because they're three blocks. We discussed different options like extending the upper arm and then shortening the lower arm just because of the way that his elbow joint would move while playing the guitar. That's a big part of the story. So, we had to ask how is that going to look appealing when you have two blocks separating and you see that joint in there. It needs to maintain the design, but it also needs to serve its function. We just wanted to make sure that things didn't bend too much, but we still had a little squash and stretch in the animation."
The need to balance the character's flexibility with their wooden appearance also posed problems for lighting the characters. Schillaci explains, "We had to do some modifications on some of the edge of the wood to be sure they would receive a good lighting. At first, we went with a very choppy wood on the face, and that was almost impossible to light. To get a nice rim on the edge, we started creating some bevels inside the edge of the face. We had to do some of that to have the more theatrical lighting that the director wanted."
[caption id="attachment_33793" align="alignnone" width="800"]
Manolo in the bullring. (Twentieth Century Fox & Reel FX)[/caption]
Lighting wasn't the only challenge the Reel FX team had to overcome. Surfacing was another key area of focus to creating the visual style of the film. According to Schillaci, between 80% to 90% of the textures in the film were hand-painted. It would be up to the CG team, lead by Glo Minaya, to solve the problem of creating workable textures. "In surfacing, we really wanted there to be a truth in the material being that the characters were made of wood. That presented some challenges when it came to deforming the geometry, especially facially when characters were speaking."
Minaya explained the CG teams' solution required finding a balance between the material's flexibility and its realism. "If you have wood and you deform something too much, then you see the wood stretch and it doesn't look good. So we had to find a balance between softening the texture and also finding a balancing point between how much we would deform the face." The solution came through a rig that was built to mimic the real-world movements and articulations of actual wooden toys. The team had to study what types of movements were actually possible and apply those to the CG model rigging.
Animators collaborated closely with Gutierrez, himself an animator, to refine the character's performances and bring his vision to life. To help make this possible, the team used Maya for rigging and animation. In the image below, you can see Guiterrez's concept drawing for Manolo (left), Sullivan's interpretation (center) and the final image of the character (right).
An example of the use of depth of field. (Twentieth Century Fox & Reel FX)[/caption]
Schillaci added that vignettes and masks were also used. "We spent a month in [Digital Intermediate] going back and forth during the process and did a lot of vignettes. That helped us out a lot. We vignetted with colorization and added some darker vignettes to focus where we wanted the viewer's eye. That's something Jorge really, really wanted. We spent a month on that. We wouldn't normally spend a month on the DI process."
The conversion to 3D also affected much of the art direction and its emphasis on creating a flat Cubist, Picasso-esq look. "We tried to find different ways to maintain the integrity of that design while in 3D," explains Sullivan, "From an art direction standpoint, one of the things I did was put like a kink in an oval if it was a solid shaped tube like, for example, in Manolo's collar and on his waist. If it was solid, we'd put a kink at the top (so an apex). From a front perspective, you would see the top of that apex and it would look flat. But then it would lean down and create a half circle. So it would look like a Cubist half-oval kind of things. Then for a lot of the set dressing items, we kept that same thing. If it was a hallow object, we just literally flattened the back of it on purpose so we could maintain that Cubist style."
Few animated features can sustain the type of rapid fire editing style that can be found in many of The Book of Life sequences. At times, the film is so action intensive that images seem to almost flash upon the screen. Nevertheless, the team was able to make the story line and action packed scene easy to follow.
Much of this editing style was to due to Gutierrez, who comes from a background of children's animated television programs that typically feature a quicker cutting style. Schillaci admits that some of the frames were no more than 15 or 16 frames long. "I think it took it to the extreme of how fast the shot could be. You would have to see the movie several times to see some of the stuff that happens during the action."
Short shot lengths, however, meant more shots overall, putting pressure on the team’s daily quotas. However, from a technical perspective, there were also benefits like being able to spread the workload among the animators. "It does help animation more than the back-end because you can distribute work to more animators," explains Schillaci. This is a different strategy to longer takes, which can take one animator much longer to create. "If it's a long shot, your quotas get a little messed up because you can't meet 50 shots per weeks. It's only one shot, but it takes a month to do. In the end, the short cuts did help us out."
Just as important, the color script served to enhance readability of the image, particularly for quickly edited scenes. "When we were in a softer, more romantic moment, we use a more broad palette of colors. But when the time frame cuts really quickly during action moments, we up the contrast and we limit the colors a little bit more so that it's a very quick read. Before we started this color script, I literally dissected the reels. I went through all of the movie and pulled from each story point of the film something that was notable in terms of the emotion, contrast and the saturation. Then I went through and literally graphed it out. So I would take each of the story points and I wrote it down and painted in the corresponding colors.
So based on a baseline I'd ask, "Is this a high emotional moment, a neutral or low emotional moment? And how does that low or high compare to the subsequent lows or highs that we get?" I wanted to make sure that I wasn't using those colors too early. If I used those colors too early, then I can't use them later on. It won't be as impactful emotionally. So that was one way how I maintained that balance where if it's a high, I brought up the colors but within a certain range. If it's a low, I brought them down within a certain range."
"When we go to the Land of the Remembered, that's a very saturated environment, very colorful and bright and contrasting. But it wouldn't feel that way if we didn't limit that saturation a little bit before that. Emotionally, when high moments happen, we want the viewer to feel happy, but we don't want to use all of our resources in making this one moment the happiest moment of the movie if later on we have a happier moment. So, if we have two or three happy moments, I'll work with Jorge to decide 'This is the happiest moment, this one's second, and this one's third.' Then I'll decide with the colors "Okay, this one is where we use all of it. We go in and put all of the saturation, all the color, all the vibrancy. At that point is when we decide 'That's what we want to show as happy.' Before that, if we have a more limited palette, I'll make sure to drop the colors a little bit more de-saturated or a little bit more muted right before that happy moment then I bring them up just a little bit during that happy moment."
Maria[/caption]
As anticipated, the director also had a clear vision of how he wanted the fire to look, which was a combination of stylization and realism. Schillaci explains, "In the beginning, he wanted the flames to be like the hot rod flames you paint on the side of a car, but we could not manage to create that look. So, we came up with an intermediate. It was definitely pretty challenging."
After the look of the fire had been nailed down, they moved onto the challenge of creating one big bull out of thousands of smaller bulls that required a little VFX smoke and mirrors to work. "We created a kind of tornado effect with the smoke. Then we created smaller pieces of bones that kind of react and rotate around that. And then as the smoke builds, we created a reverse disintegration to go from the bones into the object. But we only did it for the leg at the end. So, we didn't have to do it for the whole body."
"After that he catches fire and starts disintegrating. The effects were simple. We kind of created two different types of bulls. One we filled up with particles that were flower petals, and then we started creating the motion he wanted for the flower petals to start flying out. Then we had another bull with a set of different textures that reveals the one underneath that has the petals. We were look deving that shot almost throughout the entire movie."
Xibalba. (Twentieth Century Fox & Reel FX)[/caption]
Since Reel FX’s team was relatively small compared to most studios making a feature animated film, their strategy was to automate as much of the entire process of creating Xibalba as possible.
"We had an automated candle system," Minaya explains. "There were a couple of cool things we ultimately ended up with. For example, when animation was finished, animators would cache their character, Xibalba, and then a couple post-processes would be kicked off. One of them was to create the candle flames and actually track and lag with the animation. In Houdini they call them HDAs or OTLs, which are node networks, that would basically provide lighting with the candles and the candle geometry that could be used as a candle light. So we could actually shine light on him and it would track with the character.
"And the fur, of course, was all exported automatically. So we try to always optimize to work on things only when they need that extra special little umph," notes Minaya. "We had the ability to go in there and plus or change these objects that were being created automatically. But I would say more than 50 percent of the time the automated process was good. It worked well, it looked right, so we would let it ride."
Luckily, Reel FX’s work on Free Birds would provide them with an already R&D’ed, highly-developed wing system for adapting to Xibalba. "We tried using that," Minaya remarks,"but we ultimately settled on a system of cards that actually worked really well. The riggers did a great job on that, and the best part about it was that the system was very art directable -- those cards came directly from art," she explains. "They were like, 'This is what we want it to look like' and we were like, 'Great!' and so we were able to layer in wing designs that was exactly what the art director wanted and not just something that was a derivative of that."
As if that weren't enough complexity, Xibalba also had actual skulls for pupils. Each skull had it's own facial rig that would react differently depending on whether Xibalba was angry, upset or happy. To achieve the variations necessary, the team built a custom rig in Maya to get the flexibility they needed. However, the rigging wasn't the only challenge that the skulls created. Lighting them correctly also become a problem. "There was no traditional place to a spec hit for his light," says Minaya, "so we had to devise a little lighting system to put a little glint marker on that and how it would track with the eyes depending on where the eyes were focused."
[caption id="attachment_34211" align="alignnone" width="800"]
La Muerte. (Twentieth Century Fox & Reel FX)[/caption]
Finally, the team had to tackle the problem of the shear intricacy of Xibalba's costume and texturing. Minaya explains, "There were many layers of texturing. We had tar, we had ectoplasm -- that lava-like stuff on his neck -- we we had all of the detail that was in his shoulders and all of that had symbolism. It all had connections to story points that Jorge had and, so we stuck to it very closely. Stone materials for the shoulder pads, which hearkened back to the stone of being in the Land of the Remembered and Land of the Forgotten. And so these were all things that we really stuck to. It took a while to really perfect it, but we think it came out really well. In the end we kind of had him to the point where he wasn't a problem for us -- he was a good character -- he really flowed easily through the system."
Xibalba's compliment within the film is the sweet, feminine character La Muerte who is an homage to the La Calavera Catrina character made famous by Jose Guadalupe Posada in the early 1900's. Both Xibalba and La Muerte represent opposing sides of the story's universe, and Sullivan and company made this contrast more of an impact through the character design. "The contrast of La Muerte and Xibalba is that La Muerte is red and she has a little bit of green in her eyes. But she's sweet; she's made of sugar and she's made of everything sweet like licorice. But the only thing green on her character is in her eye shadow. Xibalba is pretty much made tar and everything kind of gross and gooey. But then he has a little bit of red in his eyes. He has some green on his gloves and some green in his design. So these two colors were symbolic of the characters coming together. Jorge always said that its showing that La Muerte only has eyes for Xibalba and Xibalba only has eyes for La Muerte even though they couldn't be further apart from each other."
[caption id="attachment_34221" align="alignnone" width="800"]
La Calavera Catrina by Jose Guadalupe Posada[/caption]
Building Different Worlds
The Book of Life contains three very different worlds: the Lands of the Living, the Remembered and the Forgotten. Each world is packed with unique visuals. For the team at Reel FX, creating these distinct settings, while still maintaining a level of continuity, was accomplished through a close collaboration across teams. Schillaci explained that in typical studios, VFX departments and art departments are often isolated from themselves and the director. But knowing that the project would be a particularly complex one, the team decided to make collaboration a priority. "The production designer, the art director and myself worked together from the beginning," Schillaci explains. "So, every time they were doing a painting or concept, they were also consulting with me. This connection between the art and 3D departments helped us know where to focus. For example, if we were defining a shot where the camera was going to go down a road, the art department would help make sure we didn't put too much stuff in the middle of the frame or that the image didn't get overly complicated." [caption id="attachment_34216" align="alignnone" width="800"] Conceptual drawing of "The Land of the Remembered" by Paul Sullivan. Twentieth Century Fox & Reel FX[/caption]
There was a general feeling among every member that bringing the director's vision to life was a top priority. Therefore, the team paid strict attention to what the art department delivered. One thing Gutierrez insisted upon was that a fidelity between the film's artwork and what was on the screen. "That was one thing Jorge really wanted. Sometimes you see the artwork for the movie, and it looks much nicer and more complex than how the movie ends up being. So, we set out from the beginning to make sure that didn't happen."
Coming up with the overall look for the film took some initial experimentation. By developing the look for a particular scene, the team could then use that look and approach it as a model for applying those characteristics to the rest of the film. One particular scene that was used for this purpose was when the character Manolo is resurrected from the dead after being bitten by snake. "That shot took us three-and-a-half months to do. With that shot we came out with a look dev of how to do all of the other shots. So, it took us longer in the beginning."
Aesthetic References
A strong collaborative relationship existed between director, Jorge Gutierrez and the film's art director, Paul Sullivan as well. Sullivan heavily prepared for interpreting Jorge's ideas of the story's settings, drawing on Spanish painters and architects heavily mixed with traditional Mexican folk art. For anyone who's seen The Book of Life already, it's clear that the film is a resembles an interesting mingling of times, places and cultures--from it's ancient Mayan-inspired architecture to its re-mixed contemporary pop songs. Sullivan explains, "[Jorge] would throw out a bunch of ideas to me and say, 'I want the Land of the Remembered to be a mix of all of these different cultures coming together and really show the history of Mexican folk art and the history of Mexico today.' I did a lot of research on that myself. I tried to visually represent some of the strong themes that are going on with not only the Day of the Dead holiday, but the history of Mexico" Visual influences included work by Spanish painter Pablo Picasso and architect Antoni Gaudi to Mexican folk graffiti art and Day of the Dead celebrations. "I looked a lot at Goudi," states Sullivan. "I looked a lot at Picasso. And I did a lot of research on Mayan culture. I'd previously worked on Tomb Raider where we went into Mayan ruins, so I had a ton of reference from that... A lot of the thought behind the basic theme went into The Land of the Remembered. We used a lot of the Mayan temples as the base and Spanish colonial structures on top of the temples because the Spanish came in and conquered the Maya." [caption id="attachment_34214" align="alignnone" width="800"] Park Guell designed by Antoni Gaudi. Photo by Bernard Gagnon[/caption]
Like many directors, Gutierrez also brought to the project many specific personal references for the film’s story and visuals. Schillaci explained some of these artistic allusions, many of which seem to make for some strange artistic bedfellows. "Jorge is very old school when it comes to the movies. He incorporated a lot of old references within it. For example, he wanted the Land of Remember to be kind of like the city in the film Metropolis. He wanted the city to feel infinite. Of course, in Metropolis, you end up having a ground, but he gave us the reference of Metropolis for how we should structure the city itself.”
Another aesthetic influence on the film's look included famous Italian director Sergio Leone and his "spaghetti westerns." Much of the color profile for the Book of Life's town of San Angel was intended as an homage to Leone's films. This included creating a sepia-toned pallet for the town scenes. Art Director Paul Sullivan gave us some insights into how this approach worked. "When they shot those [spaghetti westerns], they would shoot day-for-night and would put a night filter on top of it. So we kept that in mind with our coloring and lighting." Sullivan also explained that the film's frame narrative design (i.e. a story within a story) was inspired by The Princess Bride. "In the beginning of the Princess Bride there's a narrator who's reading the book to the kid, and we do the same thing in our movie.
Gutierrez's personal references for his vision of the film also extended to video games and other pop culture references. At one point the director wanted a particular scene featuring Joaquin and a rooftop battle with the Banditos to resemble the video game Street Fighter. "It's not like the animators came up with that idea. Some of that really was Jorge putting in a lot of different references. Like for example, Xibalba's castle...the one in the lava... was a reference to Snake Mountain from Masters of the Universe, the He-man 80's cartoon," recalls Schillaci.
Developing the Characters
The attention to details and references to Mexican/Spanish culture also translated into character design. Sullivan informed us that the main character's ancestors were references to Mexican cultural history. "When we meet up with the ancestors, they're all [Manolo's] ancestors through the generations. When we first meet up, we start with the Mayan Aztec ancestor, which is Carmelo (voiced by Jorge himself). And then we go to the Spanish Conquistador, and then we get to the modern day Mexican ancestors who were all bullfighters. It's showing that history based on those ancestors." Sullivan's collaboration with Character Designer, Sandra Equihua and Lighting Supervisor, Liz Hemme, were integral to creating a style of Mexican hand-crafted folk art for the look of the characters. This involved many small details that suggested the characters resembled actual hand-carved dolls. "We put a lot of effort into just those subtleties you see in the characters. The scuffing up on the joints as the paint would kind of erode. Little wonky cuts that you would see from an imperfect artisan as his knife carved out the edge of the shoulder or the leg. So wonkiness, asymmetry and imperfection was perfection in our art direction." [caption id="attachment_34218" align="alignnone" width="800"] Initial sketches of Manolo's arm. Twentieth Century Fox & Reel FX[/caption]
However, creating a doll-like, folk art look posed some problems for getting believable animations. Sullivan explains how one aspect of the character's arm structure presented issues. "Manolo's arms were extremely difficult to figure out because they're three blocks. We discussed different options like extending the upper arm and then shortening the lower arm just because of the way that his elbow joint would move while playing the guitar. That's a big part of the story. So, we had to ask how is that going to look appealing when you have two blocks separating and you see that joint in there. It needs to maintain the design, but it also needs to serve its function. We just wanted to make sure that things didn't bend too much, but we still had a little squash and stretch in the animation."
The need to balance the character's flexibility with their wooden appearance also posed problems for lighting the characters. Schillaci explains, "We had to do some modifications on some of the edge of the wood to be sure they would receive a good lighting. At first, we went with a very choppy wood on the face, and that was almost impossible to light. To get a nice rim on the edge, we started creating some bevels inside the edge of the face. We had to do some of that to have the more theatrical lighting that the director wanted."
[caption id="attachment_33793" align="alignnone" width="800"] Manolo in the bullring. (Twentieth Century Fox & Reel FX)[/caption]
Lighting wasn't the only challenge the Reel FX team had to overcome. Surfacing was another key area of focus to creating the visual style of the film. According to Schillaci, between 80% to 90% of the textures in the film were hand-painted. It would be up to the CG team, lead by Glo Minaya, to solve the problem of creating workable textures. "In surfacing, we really wanted there to be a truth in the material being that the characters were made of wood. That presented some challenges when it came to deforming the geometry, especially facially when characters were speaking."
Minaya explained the CG teams' solution required finding a balance between the material's flexibility and its realism. "If you have wood and you deform something too much, then you see the wood stretch and it doesn't look good. So we had to find a balance between softening the texture and also finding a balancing point between how much we would deform the face." The solution came through a rig that was built to mimic the real-world movements and articulations of actual wooden toys. The team had to study what types of movements were actually possible and apply those to the CG model rigging.
Animators collaborated closely with Gutierrez, himself an animator, to refine the character's performances and bring his vision to life. To help make this possible, the team used Maya for rigging and animation. In the image below, you can see Guiterrez's concept drawing for Manolo (left), Sullivan's interpretation (center) and the final image of the character (right).
Telling the Day of the Dead Story
While the CG team worked on fine tuning the practical matter of character animations and textures, a deeper issue surrounding the appropriate way to represent the film’s darker themes was a constant concern. Remaining faithful to the Day of the Dead celebration with its imagery of skulls, death, underworlds and ghosts was an issue that needed some careful thought given the young age of the film’s target audience. However, there couldn't be a one-size-fits-all solution to the problem. Instead, the team took multiple approaches to ensure the movie remained appropriate for its audience. One solution was in only suggesting certain aspects of the story rather than explicitly showing them. Schillaci explained that this worked within the scene with the Banditos by placing only cartridges on their bandoleers instead of actual bullets. Also, they downplayed the original plan to have the Banditos with "hand cannons". "Originally, their hands are also supposed to be cannons. And at one point they were going to shoot, but then we backed off of that because we didn't want to make it too aggressive. We kind of left the shooting only as a visual thing. Then the Banditos could still look really menacing, but we didn't go into any of their violent actions." Keeping things non-aggressive also made its way into the use of lighting as well as affecting the form and design of certain objects. For example, Sullivan explained how the Calaveras and Mexican folk art skulls within the Land of the Remembered were designed with a slightly upturned mouth to make them smile. Even the skulls teeth were rounded off to make them appear less sharp and menacing. "The shape language for the Land of the Remembered was very round, and we used that philosophy from the beginning to show in a visceral way that round things can't hurt you," Sullivan explains. For Minaya and the CG department, staying true to Gutierrez’s vision was a priority, but that sometimes collaborative solutions were needed."It was a very collaborative process," she states, "We had a lot of wonderful art, and we were all in love with it. So we didn't really feel like we needed to change it a lot. But things will always change a little bit when they’re developed in a digital medium. But we really wanted to stick as close to the artwork as possible.Maintaining Readability
With so much detail in any given shot of the film, one important focus for the team was making sure the audience paid attention to the most important details. This concern extended both to the shots' composition and to the quick editing style that dominated much of The Book of Life. Thrown into the mix was the fact that the film would also be stereoscoped for 3D versions. "Generally speaking, with any kind of animation you try to play against the foreground against the background to make things read right," Minaya explains, ”so you might take the background down or you might reduce the detail in the background. However, we really couldn't use any of those tricks for this movie since we needed to preserve all the detail" As a solution, depth of field was used extensively to direct the audience's eye. Supplementing this approach was the use of fog and haze within sequences and settings like the Land of the Remembered. "This was actually part of the design, but it ultimately helped us with readability and keeping things focused," stated Minaya. [caption id="attachment_33797" align="alignnone" width="800"] An example of the use of depth of field. (Twentieth Century Fox & Reel FX)[/caption]
Schillaci added that vignettes and masks were also used. "We spent a month in [Digital Intermediate] going back and forth during the process and did a lot of vignettes. That helped us out a lot. We vignetted with colorization and added some darker vignettes to focus where we wanted the viewer's eye. That's something Jorge really, really wanted. We spent a month on that. We wouldn't normally spend a month on the DI process."
The conversion to 3D also affected much of the art direction and its emphasis on creating a flat Cubist, Picasso-esq look. "We tried to find different ways to maintain the integrity of that design while in 3D," explains Sullivan, "From an art direction standpoint, one of the things I did was put like a kink in an oval if it was a solid shaped tube like, for example, in Manolo's collar and on his waist. If it was solid, we'd put a kink at the top (so an apex). From a front perspective, you would see the top of that apex and it would look flat. But then it would lean down and create a half circle. So it would look like a Cubist half-oval kind of things. Then for a lot of the set dressing items, we kept that same thing. If it was a hallow object, we just literally flattened the back of it on purpose so we could maintain that Cubist style."
Few animated features can sustain the type of rapid fire editing style that can be found in many of The Book of Life sequences. At times, the film is so action intensive that images seem to almost flash upon the screen. Nevertheless, the team was able to make the story line and action packed scene easy to follow.
Much of this editing style was to due to Gutierrez, who comes from a background of children's animated television programs that typically feature a quicker cutting style. Schillaci admits that some of the frames were no more than 15 or 16 frames long. "I think it took it to the extreme of how fast the shot could be. You would have to see the movie several times to see some of the stuff that happens during the action."
Short shot lengths, however, meant more shots overall, putting pressure on the team’s daily quotas. However, from a technical perspective, there were also benefits like being able to spread the workload among the animators. "It does help animation more than the back-end because you can distribute work to more animators," explains Schillaci. This is a different strategy to longer takes, which can take one animator much longer to create. "If it's a long shot, your quotas get a little messed up because you can't meet 50 shots per weeks. It's only one shot, but it takes a month to do. In the end, the short cuts did help us out."
Color Scripting The Book of Life
One important way the Reel FX art department also helped maintain readability was in a creating and referencing an effective color script. These types of chromatic reference systems are often used by studios to create a visual representation of the story's emotional language written in a variety of hues. One important function of color scripts are to help provide a road map for shifts in color, lighting, emotion and mood throughout a film. These shifts can be timed to coincide with particularly emotional moments within a story where a specific color can be matched with a corresponding scene, reinforcing its emotional impact. Art director Paul Sullivan explain how he worked with Jorge to create a precise color script for the film. "I worked with Jorge very closely to develop a color script that coincided with the story almost point for point. As the story was changing and evolving, the color script would change and evolve. It's like a giant living painting and consisted of about 126 keys that we would shift around." Removing scenes from the script would therefore have an affect on the color script as well. "It's a constant way to look at the movie as a whole and how it's affecting the viewer emotionally. It's a big study on the psychology of color, the emotion of color, lighting, and mood. Then we use each of those key points, once it's locked down, to dictate what the lighting keys look like. When we do the lighting keys, we literally pick shots that will show off the lighting. We'll do one for set lighting or one for key lighting shots. It starts to get more and more involved, but the color script serves as an initial jumping off point. To start those keys and really get the broad stroke psychology of the colors and lighting that we can then maintain. Then base those lighting keys on each of the points in the color script."
Just as important, the color script served to enhance readability of the image, particularly for quickly edited scenes. "When we were in a softer, more romantic moment, we use a more broad palette of colors. But when the time frame cuts really quickly during action moments, we up the contrast and we limit the colors a little bit more so that it's a very quick read. Before we started this color script, I literally dissected the reels. I went through all of the movie and pulled from each story point of the film something that was notable in terms of the emotion, contrast and the saturation. Then I went through and literally graphed it out. So I would take each of the story points and I wrote it down and painted in the corresponding colors.
So based on a baseline I'd ask, "Is this a high emotional moment, a neutral or low emotional moment? And how does that low or high compare to the subsequent lows or highs that we get?" I wanted to make sure that I wasn't using those colors too early. If I used those colors too early, then I can't use them later on. It won't be as impactful emotionally. So that was one way how I maintained that balance where if it's a high, I brought up the colors but within a certain range. If it's a low, I brought them down within a certain range."
"When we go to the Land of the Remembered, that's a very saturated environment, very colorful and bright and contrasting. But it wouldn't feel that way if we didn't limit that saturation a little bit before that. Emotionally, when high moments happen, we want the viewer to feel happy, but we don't want to use all of our resources in making this one moment the happiest moment of the movie if later on we have a happier moment. So, if we have two or three happy moments, I'll work with Jorge to decide 'This is the happiest moment, this one's second, and this one's third.' Then I'll decide with the colors "Okay, this one is where we use all of it. We go in and put all of the saturation, all the color, all the vibrancy. At that point is when we decide 'That's what we want to show as happy.' Before that, if we have a more limited palette, I'll make sure to drop the colors a little bit more de-saturated or a little bit more muted right before that happy moment then I bring them up just a little bit during that happy moment."
Stylizing Animation
When watching The Book of Life’s wooden characters move at such extreme speeds, it’s natural to wonder if producers ever considered using stop motion work to supplement the CG. In fact, for Reel FX, stop motion animation was a viable option, at least initially. At one point, they considered using stop motion to cut down on modeling and rendering time, but ultimately decided against it because it presented some problems. Schillaci explains, "At one point early on in the movie, we toyed with the idea. The sets were so detailed that at one point we consider building them as miniatures and creating the characters in CG. Then we could track the camera and some other things to integrate them. But eventually we worked our way out of the idea." Stop motion was also considered for some characters, but the team soon realized that it just couldn't sustain the type of animation they were going for. This happened especially with the characters Xibalba and La Muerte given how deformable they were. On some of the other wooden characters, like Manolo, it would have been a little easier, but then you would have to be mixing from one style of animation to the other one," states Schillaci. Like other CG/VFX teams, the Reel FX team relied heavily on references. This included examples from the voice actor's performances to some internally created ones like the studio's Looney Toons project. "That project really taught us a lot about how to create stretchable characters. So we built a lot of flexibility into The Book of Life's rigs in case we needed it. And in some cases we did use it, if something got really cartoony." Minaya recalls.
From Bulls to Petals
One of the most visually appealing shots of the movie involves the story’s hero, Manolo, who is thrown into a bull fighting arena with a thousand bull skeletons. After only a short battle, the horde suddenly coalesces into one gigantic, fire-breathing Toro bent on destroying Manolo in a David vs. Goliath showdown. After Manolo soothes the beast with his songs, the giant bull slowly turns into an enormous cloud made of thousands of red flower petals. Schillaci recalls his initial reaction to the director’s explanation of this shot and how the team approached it. "You have to imagine...Jorge comes into one of the rooms. He's pitching this scene to us, and it’s not just ‘Yeah, you know we're gonna have a bull and he's gonna catch on fire.’ No, it’s like we're gonna have a bull that is going to be created from a thousand bulls. And then from that he's gonna catch on fire, and then from that he's gonna disintegrate into flower petals.' I had a heart attack, and then I had to recover from that [laughing]." After recovering, Augusto and his team broke down the shot, beginning with tackling the look of the fire. Schillaci explains, "The first thing we did was start working on the fire because we knew that it was going to be used in a lot of shots. So, we did a lot of testing of the bull catching on fire, how the texture changed underneath. We made it a little more charcoal and darker so the fire would pop better, so that it will read better on top of the character." [caption id="attachment_34328" align="alignnone" width="800"] Maria[/caption]
As anticipated, the director also had a clear vision of how he wanted the fire to look, which was a combination of stylization and realism. Schillaci explains, "In the beginning, he wanted the flames to be like the hot rod flames you paint on the side of a car, but we could not manage to create that look. So, we came up with an intermediate. It was definitely pretty challenging."
After the look of the fire had been nailed down, they moved onto the challenge of creating one big bull out of thousands of smaller bulls that required a little VFX smoke and mirrors to work. "We created a kind of tornado effect with the smoke. Then we created smaller pieces of bones that kind of react and rotate around that. And then as the smoke builds, we created a reverse disintegration to go from the bones into the object. But we only did it for the leg at the end. So, we didn't have to do it for the whole body."
"After that he catches fire and starts disintegrating. The effects were simple. We kind of created two different types of bulls. One we filled up with particles that were flower petals, and then we started creating the motion he wanted for the flower petals to start flying out. Then we had another bull with a set of different textures that reveals the one underneath that has the petals. We were look deving that shot almost throughout the entire movie."
An Intricate Villain
In a movie filled with amazing visuals, one of the most dynamic and visually complex characters is the villain Xibalba. The evil master of the Land of the Forgotten had elaborate clothing and intricate aspects of his animation to contend with. One of the elements that would prove to be one of the biggest challenges was his mouth rig. Minaya explained, for example, that the director wanted Xibalba's teeth to animate in very specific ways. "He wanted him to be able to have pointy teeth when he was angry and square teeth when he was being playful; and he wanted them to be able to switch back and forth. That required quite a bit of effort to try and figure out how to create the rig." We ultimately ended up creating two separate mouth sets with different controls for both. But then we had to figure out how to transfer the vectors - the velocity vectors - from one set to another in order to make sure the motion blur was intact. That's because the point count was changing as the geometry changed. Also we wanted to get the textures to be seamless. Sometimes with geometry switching textures, you can see seams and pops." Because of the level of detail that they needed to portray in Xibalba’s movements, the final rig ended up having over 950 controls. "Everything had to be silhouetted towards camera in order to really get it to be able to read which caused the animators a lot of hand tweaking," says Minaya. "You see him and he has these ornate shoulders and horns coming out of his shoulders and the candles and the way his face was shaped. Everything had to be tweaked to really get that to read well toward the camera." [caption id="attachment_34206" align="alignnone" width="800"] Xibalba. (Twentieth Century Fox & Reel FX)[/caption]
Since Reel FX’s team was relatively small compared to most studios making a feature animated film, their strategy was to automate as much of the entire process of creating Xibalba as possible.
"We had an automated candle system," Minaya explains. "There were a couple of cool things we ultimately ended up with. For example, when animation was finished, animators would cache their character, Xibalba, and then a couple post-processes would be kicked off. One of them was to create the candle flames and actually track and lag with the animation. In Houdini they call them HDAs or OTLs, which are node networks, that would basically provide lighting with the candles and the candle geometry that could be used as a candle light. So we could actually shine light on him and it would track with the character.
"And the fur, of course, was all exported automatically. So we try to always optimize to work on things only when they need that extra special little umph," notes Minaya. "We had the ability to go in there and plus or change these objects that were being created automatically. But I would say more than 50 percent of the time the automated process was good. It worked well, it looked right, so we would let it ride."
Luckily, Reel FX’s work on Free Birds would provide them with an already R&D’ed, highly-developed wing system for adapting to Xibalba. "We tried using that," Minaya remarks,"but we ultimately settled on a system of cards that actually worked really well. The riggers did a great job on that, and the best part about it was that the system was very art directable -- those cards came directly from art," she explains. "They were like, 'This is what we want it to look like' and we were like, 'Great!' and so we were able to layer in wing designs that was exactly what the art director wanted and not just something that was a derivative of that."
As if that weren't enough complexity, Xibalba also had actual skulls for pupils. Each skull had it's own facial rig that would react differently depending on whether Xibalba was angry, upset or happy. To achieve the variations necessary, the team built a custom rig in Maya to get the flexibility they needed. However, the rigging wasn't the only challenge that the skulls created. Lighting them correctly also become a problem. "There was no traditional place to a spec hit for his light," says Minaya, "so we had to devise a little lighting system to put a little glint marker on that and how it would track with the eyes depending on where the eyes were focused."
[caption id="attachment_34211" align="alignnone" width="800"] La Muerte. (Twentieth Century Fox & Reel FX)[/caption]
Finally, the team had to tackle the problem of the shear intricacy of Xibalba's costume and texturing. Minaya explains, "There were many layers of texturing. We had tar, we had ectoplasm -- that lava-like stuff on his neck -- we we had all of the detail that was in his shoulders and all of that had symbolism. It all had connections to story points that Jorge had and, so we stuck to it very closely. Stone materials for the shoulder pads, which hearkened back to the stone of being in the Land of the Remembered and Land of the Forgotten. And so these were all things that we really stuck to. It took a while to really perfect it, but we think it came out really well. In the end we kind of had him to the point where he wasn't a problem for us -- he was a good character -- he really flowed easily through the system."
Xibalba's compliment within the film is the sweet, feminine character La Muerte who is an homage to the La Calavera Catrina character made famous by Jose Guadalupe Posada in the early 1900's. Both Xibalba and La Muerte represent opposing sides of the story's universe, and Sullivan and company made this contrast more of an impact through the character design. "The contrast of La Muerte and Xibalba is that La Muerte is red and she has a little bit of green in her eyes. But she's sweet; she's made of sugar and she's made of everything sweet like licorice. But the only thing green on her character is in her eye shadow. Xibalba is pretty much made tar and everything kind of gross and gooey. But then he has a little bit of red in his eyes. He has some green on his gloves and some green in his design. So these two colors were symbolic of the characters coming together. Jorge always said that its showing that La Muerte only has eyes for Xibalba and Xibalba only has eyes for La Muerte even though they couldn't be further apart from each other."
[caption id="attachment_34221" align="alignnone" width="800"] La Calavera Catrina by Jose Guadalupe Posada[/caption]
Tools of the Trade
Bringing The Book of Life to the big screen took a wide range of software and custom-made tools. Minaya explained what software was used and for what purpose. "ZBrush and MODO were widely used by modelers for the movie. MARI was used for texturing alongside Photoshop, which was really used liberally throughout all aspects of the pipeline as needed. Rigging and animation were done in Maya, with Houdini being used for the back-end, which included lighting, shading and rendering with Houdini's Mantra renderer." Schillaci explained what motivated the change from RenderMan, the main renderer used on Free Birds, to Mantra for the Book of Life project. "Part of that was we wanted to have the freedom and connection with the effects. This movie was very complex on FX, too. We used Houdini for our population tools like when we populated the cemetery with bread, candles, grass, wheat, and for crowd simulations in the arena. Normally we use Houdini for effects or pre-render, but in this case we needed the lighting to be very well connected with that. So that was one of our biggest reasons to switch." "That was quite a feat and we had an amazing collaboration with Side FX," Minaya comments. "They were awesome and wonderful, and we really enjoyed that process working with them; they're a great team." Everything was composited in NUKE and Qube was used for render management. Minaya notes, "We're a smaller studio, and we don't have a lot of the big, giant budgets that a lot of the larger studios have. But we really wanted this movie to look like it was a big-budget movie. To make sure we were very efficient and allowed the artists to be artists, we developed some really cool tools." Reel FX's technical department eventually developed some custom tools for helping the team hit the quotas they needed.
The process of building these tools was far from an afterthought. “We sat down and did some big white board sessions in development on how we were going to do this. It was like a separate production that was in conjunction with the film. We were like, ‘We need this tool by this date so we can hit this so we can start production on this.'”
“We developed a really cool system where we could pre-render all of the bits and pieces that make up a lighting shot prior to rendering from a simple command line,” Minaya recalls. “And so with very little effort, we could look at all of the assets, all of their looks and effects, VFX, matte painting, and everything else tied together. We could even use environment lights. We could get a pre-look at everything, fix any technical issues we saw, and then the lighters could hit the ground running.”
Minaya continues, “”You know a lot of these sets were massive–The Land of the Remembered was huge, The cemetery was very heavy because every grave had thousands of petals and candles and hundred pieces of set dressing. So we built an automated system of culling based on the camera frustum and the distance from the camera would set the level of detail for both geometry and displacement.”
The process of building these tools was far from an afterthought. “When we started this show we wanted to build some new tools,” Minaya says. “And so we sat down and did some big white board sessions in development on how we were going to do this. It was like a separate production that was in conjunction with the movie production.
“It’s not always the optimal thing, but in some instances, we were actually in production developing these tools,” admits Minaya. “But we knew we needed them. And when it came to the point where we really needed that quantity footage, we could hit it because of these tools. So we had some very talented people writing these tools so the artists could stay focused on the art and not get lost in the technical. Our developers were tremendous.”
A Labor of Love
When we asked what the best part of working on the show was, Minaya didn’t hesitate with her reply. “I would say by far it was the team. We had a lot of good-hearted people working on this team, and we encouraged each other to work really collaboratively. That brought a lot out of everyone, because everyone felt like they were a part of it – their voices were heard.”
“I think in general, in animation, and really any kind of work environment, that’s a really positive thing. I feel really privileged to have been able to be on this team. We had access to the director and art director and all of that made for an amazing show.”
“When you’re working on a show like The Book of Life that’s ultimately about love, it’s great when you’re really having such a wonderful working relationship with your co-workers. As an artist I always believe that who you are and what you’re feeling when you’re doing your art is what gets infused into your work. That was really reflected for us in what we did. We were working really well together and that was just a joy – a real joy.”