
From Imagination to Pixel: Behind the Scenes of The Book of Life
Building Different Worlds
The Book of Life contains three very different worlds: the Lands of the Living, the Remembered and the Forgotten. Each world is packed with unique visuals. For the team at Reel FX, creating these distinct settings, while still maintaining a level of continuity, was accomplished through a close collaboration across teams. Schillaci explained that in typical studios, VFX departments and art departments are often isolated from themselves and the director. But knowing that the project would be a particularly complex one, the team decided to make collaboration a priority. "The production designer, the art director and myself worked together from the beginning," Schillaci explains. "So, every time they were doing a painting or concept, they were also consulting with me. This connection between the art and 3D departments helped us know where to focus. For example, if we were defining a shot where the camera was going to go down a road, the art department would help make sure we didn't put too much stuff in the middle of the frame or that the image didn't get overly complicated." [caption id="attachment_34216" align="alignnone" width="800"]
Aesthetic References
A strong collaborative relationship existed between director, Jorge Gutierrez and the film's art director, Paul Sullivan as well. Sullivan heavily prepared for interpreting Jorge's ideas of the story's settings, drawing on Spanish painters and architects heavily mixed with traditional Mexican folk art. For anyone who's seen The Book of Life already, it's clear that the film is a resembles an interesting mingling of times, places and cultures--from it's ancient Mayan-inspired architecture to its re-mixed contemporary pop songs. Sullivan explains, "[Jorge] would throw out a bunch of ideas to me and say, 'I want the Land of the Remembered to be a mix of all of these different cultures coming together and really show the history of Mexican folk art and the history of Mexico today.' I did a lot of research on that myself. I tried to visually represent some of the strong themes that are going on with not only the Day of the Dead holiday, but the history of Mexico" Visual influences included work by Spanish painter Pablo Picasso and architect Antoni Gaudi to Mexican folk graffiti art and Day of the Dead celebrations. "I looked a lot at Goudi," states Sullivan. "I looked a lot at Picasso. And I did a lot of research on Mayan culture. I'd previously worked on Tomb Raider where we went into Mayan ruins, so I had a ton of reference from that... A lot of the thought behind the basic theme went into The Land of the Remembered. We used a lot of the Mayan temples as the base and Spanish colonial structures on top of the temples because the Spanish came in and conquered the Maya." [caption id="attachment_34214" align="alignnone" width="800"]
Developing the Characters
The attention to details and references to Mexican/Spanish culture also translated into character design. Sullivan informed us that the main character's ancestors were references to Mexican cultural history. "When we meet up with the ancestors, they're all [Manolo's] ancestors through the generations. When we first meet up, we start with the Mayan Aztec ancestor, which is Carmelo (voiced by Jorge himself). And then we go to the Spanish Conquistador, and then we get to the modern day Mexican ancestors who were all bullfighters. It's showing that history based on those ancestors." Sullivan's collaboration with Character Designer, Sandra Equihua and Lighting Supervisor, Liz Hemme, were integral to creating a style of Mexican hand-crafted folk art for the look of the characters. This involved many small details that suggested the characters resembled actual hand-carved dolls. "We put a lot of effort into just those subtleties you see in the characters. The scuffing up on the joints as the paint would kind of erode. Little wonky cuts that you would see from an imperfect artisan as his knife carved out the edge of the shoulder or the leg. So wonkiness, asymmetry and imperfection was perfection in our art direction." [caption id="attachment_34218" align="alignnone" width="800"]


Telling the Day of the Dead Story
While the CG team worked on fine tuning the practical matter of character animations and textures, a deeper issue surrounding the appropriate way to represent the film’s darker themes was a constant concern. Remaining faithful to the Day of the Dead celebration with its imagery of skulls, death, underworlds and ghosts was an issue that needed some careful thought given the young age of the film’s target audience. However, there couldn't be a one-size-fits-all solution to the problem. Instead, the team took multiple approaches to ensure the movie remained appropriate for its audience. One solution was in only suggesting certain aspects of the story rather than explicitly showing them. Schillaci explained that this worked within the scene with the Banditos by placing only cartridges on their bandoleers instead of actual bullets. Also, they downplayed the original plan to have the Banditos with "hand cannons". "Originally, their hands are also supposed to be cannons. And at one point they were going to shoot, but then we backed off of that because we didn't want to make it too aggressive. We kind of left the shooting only as a visual thing. Then the Banditos could still look really menacing, but we didn't go into any of their violent actions." Keeping things non-aggressive also made its way into the use of lighting as well as affecting the form and design of certain objects. For example, Sullivan explained how the Calaveras and Mexican folk art skulls within the Land of the Remembered were designed with a slightly upturned mouth to make them smile. Even the skulls teeth were rounded off to make them appear less sharp and menacing. "The shape language for the Land of the Remembered was very round, and we used that philosophy from the beginning to show in a visceral way that round things can't hurt you," Sullivan explains. For Minaya and the CG department, staying true to Gutierrez’s vision was a priority, but that sometimes collaborative solutions were needed."It was a very collaborative process," she states, "We had a lot of wonderful art, and we were all in love with it. So we didn't really feel like we needed to change it a lot. But things will always change a little bit when they’re developed in a digital medium. But we really wanted to stick as close to the artwork as possible.Maintaining Readability
With so much detail in any given shot of the film, one important focus for the team was making sure the audience paid attention to the most important details. This concern extended both to the shots' composition and to the quick editing style that dominated much of The Book of Life. Thrown into the mix was the fact that the film would also be stereoscoped for 3D versions. "Generally speaking, with any kind of animation you try to play against the foreground against the background to make things read right," Minaya explains, ”so you might take the background down or you might reduce the detail in the background. However, we really couldn't use any of those tricks for this movie since we needed to preserve all the detail" As a solution, depth of field was used extensively to direct the audience's eye. Supplementing this approach was the use of fog and haze within sequences and settings like the Land of the Remembered. "This was actually part of the design, but it ultimately helped us with readability and keeping things focused," stated Minaya. [caption id="attachment_33797" align="alignnone" width="800"]

Color Scripting The Book of Life
One important way the Reel FX art department also helped maintain readability was in a creating and referencing an effective color script. These types of chromatic reference systems are often used by studios to create a visual representation of the story's emotional language written in a variety of hues. One important function of color scripts are to help provide a road map for shifts in color, lighting, emotion and mood throughout a film. These shifts can be timed to coincide with particularly emotional moments within a story where a specific color can be matched with a corresponding scene, reinforcing its emotional impact. Art director Paul Sullivan explain how he worked with Jorge to create a precise color script for the film. "I worked with Jorge very closely to develop a color script that coincided with the story almost point for point. As the story was changing and evolving, the color script would change and evolve. It's like a giant living painting and consisted of about 126 keys that we would shift around." Removing scenes from the script would therefore have an affect on the color script as well. "It's a constant way to look at the movie as a whole and how it's affecting the viewer emotionally. It's a big study on the psychology of color, the emotion of color, lighting, and mood. Then we use each of those key points, once it's locked down, to dictate what the lighting keys look like. When we do the lighting keys, we literally pick shots that will show off the lighting. We'll do one for set lighting or one for key lighting shots. It starts to get more and more involved, but the color script serves as an initial jumping off point. To start those keys and really get the broad stroke psychology of the colors and lighting that we can then maintain. Then base those lighting keys on each of the points in the color script."
Stylizing Animation
When watching The Book of Life’s wooden characters move at such extreme speeds, it’s natural to wonder if producers ever considered using stop motion work to supplement the CG. In fact, for Reel FX, stop motion animation was a viable option, at least initially. At one point, they considered using stop motion to cut down on modeling and rendering time, but ultimately decided against it because it presented some problems. Schillaci explains, "At one point early on in the movie, we toyed with the idea. The sets were so detailed that at one point we consider building them as miniatures and creating the characters in CG. Then we could track the camera and some other things to integrate them. But eventually we worked our way out of the idea." Stop motion was also considered for some characters, but the team soon realized that it just couldn't sustain the type of animation they were going for. This happened especially with the characters Xibalba and La Muerte given how deformable they were. On some of the other wooden characters, like Manolo, it would have been a little easier, but then you would have to be mixing from one style of animation to the other one," states Schillaci. Like other CG/VFX teams, the Reel FX team relied heavily on references. This included examples from the voice actor's performances to some internally created ones like the studio's Looney Toons project. "That project really taught us a lot about how to create stretchable characters. So we built a lot of flexibility into The Book of Life's rigs in case we needed it. And in some cases we did use it, if something got really cartoony." Minaya recalls.
From Bulls to Petals
One of the most visually appealing shots of the movie involves the story’s hero, Manolo, who is thrown into a bull fighting arena with a thousand bull skeletons. After only a short battle, the horde suddenly coalesces into one gigantic, fire-breathing Toro bent on destroying Manolo in a David vs. Goliath showdown. After Manolo soothes the beast with his songs, the giant bull slowly turns into an enormous cloud made of thousands of red flower petals. Schillaci recalls his initial reaction to the director’s explanation of this shot and how the team approached it. "You have to imagine...Jorge comes into one of the rooms. He's pitching this scene to us, and it’s not just ‘Yeah, you know we're gonna have a bull and he's gonna catch on fire.’ No, it’s like we're gonna have a bull that is going to be created from a thousand bulls. And then from that he's gonna catch on fire, and then from that he's gonna disintegrate into flower petals.' I had a heart attack, and then I had to recover from that [laughing]." After recovering, Augusto and his team broke down the shot, beginning with tackling the look of the fire. Schillaci explains, "The first thing we did was start working on the fire because we knew that it was going to be used in a lot of shots. So, we did a lot of testing of the bull catching on fire, how the texture changed underneath. We made it a little more charcoal and darker so the fire would pop better, so that it will read better on top of the character." [caption id="attachment_34328" align="alignnone" width="800"]
An Intricate Villain
In a movie filled with amazing visuals, one of the most dynamic and visually complex characters is the villain Xibalba. The evil master of the Land of the Forgotten had elaborate clothing and intricate aspects of his animation to contend with. One of the elements that would prove to be one of the biggest challenges was his mouth rig. Minaya explained, for example, that the director wanted Xibalba's teeth to animate in very specific ways. "He wanted him to be able to have pointy teeth when he was angry and square teeth when he was being playful; and he wanted them to be able to switch back and forth. That required quite a bit of effort to try and figure out how to create the rig." We ultimately ended up creating two separate mouth sets with different controls for both. But then we had to figure out how to transfer the vectors - the velocity vectors - from one set to another in order to make sure the motion blur was intact. That's because the point count was changing as the geometry changed. Also we wanted to get the textures to be seamless. Sometimes with geometry switching textures, you can see seams and pops." Because of the level of detail that they needed to portray in Xibalba’s movements, the final rig ended up having over 950 controls. "Everything had to be silhouetted towards camera in order to really get it to be able to read which caused the animators a lot of hand tweaking," says Minaya. "You see him and he has these ornate shoulders and horns coming out of his shoulders and the candles and the way his face was shaped. Everything had to be tweaked to really get that to read well toward the camera." [caption id="attachment_34206" align="alignnone" width="800"]


Tools of the Trade
Bringing The Book of Life to the big screen took a wide range of software and custom-made tools. Minaya explained what software was used and for what purpose. "ZBrush and MODO were widely used by modelers for the movie. MARI was used for texturing alongside Photoshop, which was really used liberally throughout all aspects of the pipeline as needed. Rigging and animation were done in Maya, with Houdini being used for the back-end, which included lighting, shading and rendering with Houdini's Mantra renderer." Schillaci explained what motivated the change from RenderMan, the main renderer used on Free Birds, to Mantra for the Book of Life project. "Part of that was we wanted to have the freedom and connection with the effects. This movie was very complex on FX, too. We used Houdini for our population tools like when we populated the cemetery with bread, candles, grass, wheat, and for crowd simulations in the arena. Normally we use Houdini for effects or pre-render, but in this case we needed the lighting to be very well connected with that. So that was one of our biggest reasons to switch." "That was quite a feat and we had an amazing collaboration with Side FX," Minaya comments. "They were awesome and wonderful, and we really enjoyed that process working with them; they're a great team." Everything was composited in NUKE and Qube was used for render management. Minaya notes, "We're a smaller studio, and we don't have a lot of the big, giant budgets that a lot of the larger studios have. But we really wanted this movie to look like it was a big-budget movie. To make sure we were very efficient and allowed the artists to be artists, we developed some really cool tools." Reel FX's technical department eventually developed some custom tools for helping the team hit the quotas they needed.

The process of building these tools was far from an afterthought. “We sat down and did some big white board sessions in development on how we were going to do this. It was like a separate production that was in conjunction with the film. We were like, ‘We need this tool by this date so we can hit this so we can start production on this.'”
“We developed a really cool system where we could pre-render all of the bits and pieces that make up a lighting shot prior to rendering from a simple command line,” Minaya recalls. “And so with very little effort, we could look at all of the assets, all of their looks and effects, VFX, matte painting, and everything else tied together. We could even use environment lights. We could get a pre-look at everything, fix any technical issues we saw, and then the lighters could hit the ground running.”
Minaya continues, “”You know a lot of these sets were massive–The Land of the Remembered was huge, The cemetery was very heavy because every grave had thousands of petals and candles and hundred pieces of set dressing. So we built an automated system of culling based on the camera frustum and the distance from the camera would set the level of detail for both geometry and displacement.”
The process of building these tools was far from an afterthought. “When we started this show we wanted to build some new tools,” Minaya says. “And so we sat down and did some big white board sessions in development on how we were going to do this. It was like a separate production that was in conjunction with the movie production.
“It’s not always the optimal thing, but in some instances, we were actually in production developing these tools,” admits Minaya. “But we knew we needed them. And when it came to the point where we really needed that quantity footage, we could hit it because of these tools. So we had some very talented people writing these tools so the artists could stay focused on the art and not get lost in the technical. Our developers were tremendous.”
A Labor of Love
When we asked what the best part of working on the show was, Minaya didn’t hesitate with her reply. “I would say by far it was the team. We had a lot of good-hearted people working on this team, and we encouraged each other to work really collaboratively. That brought a lot out of everyone, because everyone felt like they were a part of it – their voices were heard.”
“I think in general, in animation, and really any kind of work environment, that’s a really positive thing. I feel really privileged to have been able to be on this team. We had access to the director and art director and all of that made for an amazing show.”
“When you’re working on a show like The Book of Life that’s ultimately about love, it’s great when you’re really having such a wonderful working relationship with your co-workers. As an artist I always believe that who you are and what you’re feeling when you’re doing your art is what gets infused into your work. That was really reflected for us in what we did. We were working really well together and that was just a joy – a real joy.”