From Design to Pixel: Behind the Scenes of The Book of Life

The Texas-based VFX studio, Reel FX, was founded in 1993, but it made its first entrance into the world of in-house animated features in November 2013.  Its release was Free Birds, a quirky holiday tale of time traveling turkeys set to save Thanksgiving. Less than a year later, the studio had already released its second in-house animated movie, The Book of Life. 

The story is an ambitious retelling of the origins of the Day of the Dead and spans three completely distinct worlds. Each setting needed to have its own look and feel. More importantly, the film was a culmination of over a decade of planning by writer/director Jorge Gutierrez who already had definite ideas about how he wanted the story to be represented visually. 

A large part of The Book of Life’s success was due to its unique visual style. It was up to Reel FX’s talented, but small, team of CG artists, VFX specialists and supervisors to help bring Gutierrez’s vision to life. When the final images were rendered, the team had succeeded in incorporating a wide variety of visual styles, creating some innovative custom-built tools, and building an atmosphere based on close collaboration. 

Back when the film was released, we were given a first class tour of the studio's production by several of its supervising members: Paul Sullivan (Art Director), Augusto Schillaci (VFX Supervisor) and Glo Minaya (CG Supervisor). All three provide a unique perspective on understanding how the studio's process helped realize the director’s epic vision with only a fraction of the budget allotted to most animated features.

The Book of Life contains three very different worlds: the Lands of the Living, the Remembered, and the Forgotten. Each world is packed with unique visuals. For the team at Reel FX, creating these distinct settings, while still maintaining a level of continuity, was accomplished through a close collaboration across teams. Schillaci explained that in typical studios, VFX departments and art departments are often isolated from themselves and the director. 

But knowing that the project would be a particularly complex one, the team decided to make collaboration a priority. "The production designer, the art director and myself worked together from the beginning," Schillaci explains. "So, every time they were doing a painting or concept, they were also consulting with me. This connection between the art and 3D departments helped us know where to focus.”

There was a general feeling among every member that bringing the director's vision to life was a top priority. Therefore, the team paid strict attention to what the art department delivered.

One thing Gutierrez insisted upon was that a fidelity between the film's artwork and what was on the screen. "Sometimes you see the artwork for the movie, and it looks much nicer and more complex than how the movie ends up being. So, we set out from the beginning to make sure that didn't happen." Coming up with the overall look for the film took some initial experimentation.

By developing the look for a particular scene, the team could then use that look and approach it as a model for applying those characteristics to the rest of the film. One particular scene that was used for this purpose was when the character Manolo is resurrected from the dead after being bitten by snake. "That shot took us three-and-a-half months to do. With that shot we came out with a look dev of how to do all of the other shots. So, it took us longer in the beginning."

A strong collaborative relationship existed between director, Jorge Gutierrez and the film's art director, Paul Sullivan as well. Sullivan heavily prepared for interpreting Jorge's ideas of the story's settings, drawing on Spanish painters and architects heavily mixed with traditional Mexican folk art.

For anyone who's seen The Book of Life already, it's clear that the film is a resembles an interesting mingling of times, places and cultures--from it's ancient Mayan-inspired architecture to its re-mixed contemporary pop songs. Sullivan explains, "[Jorge] would throw out a bunch of ideas to me and say, 'I want the Land of the Remembered to be a mix of all of these different cultures coming together and really show the history of Mexican folk art and the history of Mexico today.' I did a lot of research on that myself. I tried to visually represent some of the strong themes that are going on with not only the Day of the Dead holiday, but the history of Mexico"

Visual influences included work by Spanish painter Pablo Picasso and architect Antoni Gaudi to Mexican folk graffiti art and Day of the Dead celebrations. "I looked a lot at Goudi," states Sullivan. "I looked a lot at Picasso. And I did a lot of research on Mayan culture.... We used a lot of the Mayan temples as the base and Spanish colonial structures on top of the temples because the Spanish came in and conquered the Maya."

Like many directors, Gutierrez also brought to the project many specific personal references for the film’s story and visuals. Schillaci explained some of these artistic allusions, many of which seem to make for some strange artistic bedfellows. "Jorge is very old school when it comes to the movies. He incorporated a lot of old references within it. For example, he wanted the Land of Remember to be kind of like the city in the film Metropolis. He wanted the city to feel infinite.

Another aesthetic influence on the film's look included famous Italian director Sergio Leone and his "spaghetti westerns." Much of the color profile for the Book of Life's town of San Angel was intended as an homage to Leone's films. This included creating a sepia-toned pallet for the town scenes. Art Director Paul Sullivan gave us some insights into how this approach worked. "When they shot those [spaghetti westerns], they would shoot day-for-night and would put a night filter on top of it. So we kept that in mind with our coloring and lighting."

Sullivan also explained that the film's frame narrative design (i.e. a story within a story) was inspired by The Princess Bride. "In the beginning of the Princess Bride there's a narrator who's reading the book to the kid, and we do the same thing in our movie. Gutierrez's personal references for his vision of the film also extended to video games and other pop culture references.

At one point the director wanted a particular scene featuring Joaquin and a rooftop battle with the Banditos to resemble the video game Street Fighter. "It's not like the animators came up with that idea. Some of that really was Jorge putting in a lot of different references. Like for example, Xibalba's castle...the one in the lava... was a reference to Snake Mountain from Masters of the Universe, the He-man 80's cartoon," recalls Schillaci.

The attention to details and references to Mexican/Spanish culture also translated into character design. Sullivan informed us that the main character's ancestors were references to Mexican cultural history. "When we meet up with the ancestors, they're all [Manolo's] ancestors through the generations. When we first meet up, we start with the Mayan Aztec ancestor, which is Carmelo (voiced by Jorge himself). And then we go to the Spanish Conquistador, and then we get to the modern day Mexican ancestors who were all bullfighters."

Sullivan's collaboration with Character Designer, Sandra Equihua and Lighting Supervisor, Liz Hemme, were integral to creating a style of Mexican hand-crafted folk art for the look of the characters. This involved many small details that suggested the characters resembled actual hand-carved dolls. "We put a lot of effort into just those subtleties you see in the characters. The scuffing up on the joints as the paint would kind of erode. Little wonky cuts that you would see from an imperfect artisan as his knife carved out the edge of the shoulder or the leg."

However, creating a doll-like, folk art look posed some problems for getting believable animations. Sullivan explains how one aspect of the character's arm structure presented issues. "Manolo's arms were extremely difficult to figure out because they're three blocks.... So, we had to ask how is that going to look appealing when you have two blocks separating and you see that joint in there. It needs to maintain the design, but it also needs to serve its function. We just wanted to make sure that things didn't bend too much, but we still had a little squash and stretch in the animation."

The need to balance the character's flexibility with their wooden appearance also posed problems for lighting the characters. Schillaci explains, "We had to do some modifications on some of the edge of the wood to be sure they would receive a good lighting.... To get a nice rim on the edge, we started creating some bevels inside the edge of the face. We had to do some of that to have the more theatrical lighting that the director wanted."

Lighting wasn't the only challenge the Reel FX team had to overcome. Surfacing was another key area of focus to creating the visual style of the film. According to Schillaci, between 80% to 90% of the textures in the film were hand-painted. It would be up to the CG team, lead by Glo Minaya, to solve the problem of creating workable textures. 

Minaya explained the CG teams' solution required finding a balance between the material's flexibility and its realism. "If you have wood and you deform something too much, then you see the wood stretch and it doesn't look good. So we had to find a balance between softening the texture and also finding a balancing point between how much we would deform the face."

The solution came through a rig that was built to mimic the real-world movements and articulations of actual wooden toys. The team had to study what types of movements were actually possible and apply those to the CG model rigging. Animators collaborated closely with Gutierrez, himself an animator, to refine the character's performances and bring his vision to life. To help make this possible, the team used Maya for rigging and animation. In the image below, you can see Guiterrez's concept drawing for Manolo (left), Sullivan's interpretation (center) and the final image of the character (right)

While the CG team worked on fine tuning the practical matter of character animations and textures, a deeper issue surrounding the appropriate way to represent the film’s darker themes was a constant concern. Remaining faithful to the Day of the Dead celebration with its imagery of skulls, death, underworlds and ghosts was an issue that needed some careful thought given the young age of the film’s target audience.

The team took multiple approaches to ensure the movie remained appropriate for its audience. One solution was in only suggesting certain aspects of the story rather than explicitly showing them. Schillaci explained that this worked within the scene with the Banditos by placing only cartridges on their bandoleers instead of actual bullets. Also, they downplayed the original plan to arm the Banditos with "hand cannons." 

This approach impacted the use of lighting as well as the form and design of certain objects. For example, Sullivan explained how the Calaveras and Mexican folk art skulls within the Land of the Remembered were designed with a slightly upturned mouth to make them smile. Even the skulls teeth were rounded off to make them appear less sharp and menacing. "The shape language for the Land of the Remembered was very round, and we used that philosophy from the beginning to show in a visceral way that round things can't hurt you," Sullivan explains.

With so much detail in any given shot of the film, one important focus for the team was making sure the audience paid attention to the most important details. This concern extended both to the shots' composition and to the quick editing style that dominated much of The Book of Life.

Thrown into the mix was the fact that the film would also be stereoscoped for 3D versions. "Generally speaking, with any kind of animation you try to play against the foreground against the background to make things read right," Minaya explains, ”so you might take the background down or you might reduce the detail in the background. However, we really couldn't use any of those tricks for this movie since we needed to preserve all the detail"

As a solution, depth of field was used extensively to direct the audience's eye. Supplementing this approach was the use of fog and haze within sequences and settings like the Land of the Remembered. "This was actually part of the design, but it ultimately helped us with readability and keeping things focused," stated Minaya.

Schillaci added that vignettes and masks were also used. "We spent a month in [Digital Intermediate] going back and forth during the process and did a lot of vignettes. That helped us out a lot. We vignetted with colorization and added some darker vignettes to focus where we wanted the viewer's eye."

The conversion to 3D also affected much of the art direction and its emphasis on creating a flat Cubist, Picasso-esq look. "We tried to find different ways to maintain the integrity of that design while in 3D," explains Sullivan, "From an art direction standpoint, one of the things I did was put like a kink in an oval if it was a solid shaped tube like, for example, in Manolo's collar and on his waist. If it was solid, we'd put a kink at the top (so an apex). From a front perspective, you would see the top of that apex and it would look flat. But then it would lean down and create a half circle. So it would look like a Cubist half-oval kind of things."

Few animated features can sustain the type of rapid fire editing style that can be found in many of The Book of Life sequences. At times, the film is so action intensive that images seem to almost flash upon the screen. Nevertheless, the team was able to make the story line and action packed scene easy to follow.

Much of this editing style was to due to Gutierrez, who comes from a  background of children's animated television programs that typically feature a quicker cutting style. Schillaci admits that some of the frames were no more than 15 or 16 frames long. "I think it took it to the extreme of how fast the shot could be. You would have to see the movie several times to see some of the stuff that happens during the action."

Short shot lengths, however, meant more shots overall, putting pressure on the team’s daily quotas. However, from a technical perspective, there were also benefits like being able to spread the workload among the animators. "It does help animation more than the back-end because you can distribute work to more animators," explains Schillaci. This is a different strategy to longer takes, which can take one animator much longer to create. 

One important way the Reel FX art department also helped maintain readability was in a creating and referencing an effective color script. These types of chromatic reference systems are often used by studios to create a visual representation of the story's emotional language written in a variety of hues. One important function of color scripts are to help provide a road map for shifts in color, lighting, emotion and mood throughout a film. These shifts can be timed to coincide with particularly emotional moments within a story.

Art director Paul Sullivan explains, "I worked with Jorge very closely to develop a color script that coincided with the story almost point for point. As the story was changing and evolving, the color script would change and evolve. It's like a giant living painting and consisted of about 126 keys that we would shift around."

Removing scenes from the script would therefore have an affect on the color script as well. "It's a constant way to look at the movie as a whole and how it's affecting the viewer emotionally. It's a big study on the psychology of color, the emotion of color, lighting, and mood. Then we use each of those key points, once it's locked down, to dictate what the lighting keys look like."

Just as important, the color script served to enhance readability of the image, particularly for quickly edited scenes. "When we were in a softer, more romantic moment, we use a more broad palette of colors. But when the time frame cuts really quickly during action moments, we up the contrast and we limit the colors a little bit more so that it's a very quick read. Before we started this color script, I literally dissected the reels. I went through all of the movie and pulled from each story point of the film something that was notable in terms of the emotion, contrast and the saturation. Then I went through and literally graphed it out."

"When we go to the Land of the Remembered, that's a very saturated environment, very colorful and bright and contrasting. But it wouldn't feel that way if we didn't limit that saturation a little bit before that. Emotionally, when high moments happen, we want the viewer to feel happy, but we don't want to use all of our resources in making this one moment the happiest moment of the movie if later on we have a happier moment."

When watching The Book of Life’s wooden characters move at such extreme speeds, it’s natural to wonder if producers ever considered using stop motion work to supplement the CG. In fact, for Reel FX, stop motion animation was a viable option, at least initially. At one point, they considered using stop motion to cut down on modeling and rendering time, but ultimately decided against it because it presented some problems.

Schillaci explains, "At one point early on in the movie, we toyed with the idea. The sets were so detailed that at one point we consider building them as miniatures and creating the characters in CG. Then we could track the camera and some other things to integrate them. But eventually we worked our way out of the idea." Stop motion was also considered for some characters, but the team soon realized that it just couldn't sustain the type of animation they were going for. 

Like other CG/VFX teams, the Reel FX team relied heavily on references. This included examples from the voice actor's performances to some internally created ones like the studio's Looney Toons project. "That project really taught us a lot about how to create stretchable characters. So we built a lot of flexibility into The Book of Life's rigs in case we needed it. And in some cases we did use it, if something got really cartoony." Minaya recalls.

One of the most visually appealing shots of the movie involves the story’s hero, Manolo, who is thrown into a bull fighting arena with a thousand bull skeletons. After only a short battle, the horde suddenly coalesces into one gigantic, fire-breathing Toro bent on destroying Manolo in a David vs. Goliath showdown. After Manolo soothes the beast with his songs, the giant bull slowly turns into an enormous cloud made of thousands of red flower petals.

Augusto and his team broke down the shot, beginning with the look of the fire. Schillaci explains, "The first thing we did was start working on the fire because we knew that it was going to be used in a lot of shots. So, we did a lot of testing of the bull catching on fire, how the texture changed underneath. We made it a little more charcoal and darker so the fire would pop better, so that it will read better on top of the character."

As anticipated, the director also had a clear vision of how he wanted the fire to look, which was a combination of stylization and realism. Schillaci explains, "In the beginning, he wanted the flames to be like the hot rod flames you paint on the side of a car, but we could not manage to create that look. So, we came up with an intermediate. It was definitely pretty challenging."

In a movie filled with amazing visuals, one of the most dynamic and visually complex characters is the villain Xibalba. The evil master of the Land of the Forgotten had elaborate clothing and intricate aspects of his animation to contend with. One of the elements that would prove to be one of the biggest challenges was his mouth rig.

Minaya explained, for example, that the director wanted Xibalba's teeth to animate in very specific ways. "He wanted him to be able to have pointy teeth when he was angry and square teeth when he was being playful; and he wanted them to be able to switch back and forth. That required quite a bit of effort to try and figure out how to create the rig."

Because of the level of detail that they needed to portray in Xibalba’s movements, the final rig ended up having over 950 controls. "Everything had to be silhouetted towards camera in order to really get it to be able to read which caused the animators a lot of hand tweaking," says Minaya. "You see him and he has these ornate shoulders and horns coming out of his shoulders and the candles and the way his face was shaped. Everything had to be tweaked to really get that to read well toward the camera."

Since Reel FX’s team was relatively small compared to most studios making a feature animated film, their strategy was to automate as much of the entire process of creating Xibalba as possible.

"We had an automated candle system," Minaya explains. "There were a couple of cool things we ultimately ended up with. For example, when animation was finished, animators would cache their character, Xibalba, and then a couple post-processes would be kicked off. One of them was to create the candle flames and actually track and lag with the animation. In Houdini they call them HDAs or OTLs, which are node networks, that would basically provide lighting with the candles and the candle geometry that could be used as a candle light. So we could actually shine light on him and it would track with the character. "And the fur, of course, was all exported automatically. So we try to always optimize to work on things only when they need that extra special little umph," notes Minaya.

As if that weren't enough complexity, Xibalba also had actual skulls for pupils. Each skull had it's own facial rig that would react differently depending on whether Xibalba was angry, upset or happy. To achieve the variations necessary, the team built a custom rig in Maya to get the flexibility they needed. However, the rigging wasn't the only challenge that the skulls created. Lighting them correctly also become a problem. "There was no traditional place to a spec hit for his light," says Minaya, "so we had to devise a little lighting system to put a little glint marker on that and how it would track with the eyes depending on where the eyes were focused."

Finally, the team had to tackle the problem of the shear intricacy of Xibalba's costume and texturing. Minaya explains, "There were many layers of texturing. We had tar, we had ectoplasm -- that lava-like stuff on his neck -- we we had all of the detail that was in his shoulders and all of that had symbolism. It all had connections to story points that Jorge had and, so we stuck to it very closely. Stone materials for the shoulder pads, which hearkened back to the stone of being in the Land of the Remembered and Land of the Forgotten."

Xibalba's compliment within the film is the sweet, feminine character La Muerte who is an homage to the La Calavera Catrina character made famous by Jose Guadalupe Posada in the early 1900's. Both Xibalba and La Muerte represent opposing sides of the story's universe, and Sullivan and company made this contrast more of an impact through the character design. "The contrast of La Muerte and Xibalba is that La Muerte is red and she has a little bit of green in her eyes.... [Xibalba] has some green on his gloves and some green in his design. So these two colors were symbolic of the characters coming together."

Bringing The Book of Life to the big screen took a wide range of software and custom-made tools. Minaya explained what software was used and for what purpose. "ZBrush and MODO were widely used by modelers for the movie. MARI was used for texturing alongside Photoshop, which was really used liberally throughout all aspects of the pipeline as needed. Rigging and animation were done in Maya, with Houdini being used for the back-end, which included lighting, shading and rendering with Houdini's Mantra renderer."

Everything was composited in NUKE and Qube was used for render management. Minaya notes, "We're a smaller studio, and we don't have a lot of the big, giant budgets that a lot of the larger studios have. But we really wanted this movie to look like it was a big-budget movie. To make sure we were very efficient and allowed the artists to be artists, we developed some really cool tools." Reel FX's technical department eventually developed some custom tools for helping the team hit the quotas they needed.

The process of building these tools was far from an afterthought. “We developed a really cool system where we could pre-render all of the bits and pieces that make up a lighting shot prior to rendering from a simple command line,” Minaya recalls. “And so with very little effort, we could look at all of the assets, all of their looks and effects, VFX, matte painting, and everything else tied together. We could even use environment lights. We could get a pre-look at everything, fix any technical issues we saw, and then the lighters could hit the ground running.”

Minaya continues, “”You know a lot of these sets were massive–The Land of the Remembered was huge, The cemetery was very heavy because every grave had thousands of petals and candles and hundred pieces of set dressing. So we built an automated system of culling based on the camera frustum and the distance from the camera would set the level of detail for both geometry and displacement.”

When we asked what the best part of working on The Book of Life, Minaya didn’t hesitate with her reply. “I would say by far it was the team. We had a lot of good-hearted people working on this team, and we encouraged each other to work really collaboratively. That brought a lot out of everyone, because everyone felt like they were a part of it – their voices were heard.”

“I think in general, in animation, and really any kind of work environment, that’s a really positive thing. I feel really privileged to have been able to be on this team. We had access to the director and art director and all of that made for an amazing show.”

“When you’re working on a show like The Book of Life that’s ultimately about love, it’s great when you’re really having such a wonderful working relationship with your co-workers. As an artist I always believe that who you are and what you’re feeling when you’re doing your art is what gets infused into your work. That was really reflected for us in what we did. We were working really well together and that was just a joy – a real joy.”