The Ballet Of Robotics and Motion Capture

Last year we saw the awe-inspiring results of mixing projection mapping, robotics and cinematography in the short film, Box. The film's director, Tarik Abdel-Gawad introduced us to the use of those large robotic arms you often see on automobile assemble floors. Their welding and grinding are more often associated with producing a Kia than with creating beautiful cinematography. However, Box showed that the use of these types of industrial robotics towards cinematography seemed to open up infinite possibilities for film makers and artists to explore. Recently, Abdel-Gawad has again combined the same robotics, but this time with motion capture in an experiment that uses 3D motion capture models to program a camera's motions. Abdel-Gawad's experimental film is called "Francesca Da Rimini". The 3 minute short that features a ballet performance by Maria Kochetkova and Joan Boada. As impressive as their dancing is by itself, it's Abdel-Gawad's vision of having the camera "dance" along with the actors that makes the film somewhat unique. We all know that ballet takes a lot of training and repetition to master the required steps, leaps, and lifts. It's this element of repetition and accuracy that drew the director towards capturing a ballet performance with a robotic camera. "The tradition requires that [dancers] perform moves in a specific way, states Abdel-Gawad, "In order to do that, they do it over and over and over again. So I thought let's see how this matches up with something that's really mechanical like a robotically-controlled camera." Abdel-Gawad's production flow began with capturing the motion of the ballet dancers. Next, 3D models were built around their live performance. Then the computer's camera was manipulated within the 3D space exactly where Abdel-Gawad wanted it and the viewer to be. The robotic camera was then programmed with these pre-determined angles, heights, and movements. Finally, the real dancers came back to perform as closely to the original as possible with the robotic camera following along. "Some of these shots could have been accomplished using...more traditional film techniques and a lot more rehearsal time, states Abdel-Gawad, " But that wasn't the point. We proved we could coordinate the camera around an actual dancer. Using this approach we can either take a new lens on dance in the form of a film or apply this technology on a live stage." Ballerinas in motion capture suits Certainly, Abdel-Gawad's experiment has a limited application to traditional cinematography, directing, and acting right now. But it does show his continued interest to combine different technologies in unique ways. Further development of this robotic camera style with motion capture might provide a more efficient and less-costly alternative for producers. If a film's scenes are highly staged and predictable, which most are, then directors of photography might be assisted by 3D artists, animators, and their robot side kicks. The most likely application would be in live staged performances that could bring viewers closer to the action than the traditional theater stage. Increasingly, theater chains like AMC are offering such live performance screenings of operas and ballets. Whatever its shortcomings with respect to application,"Francesca Da Rimini" does present an incredibly immersive experience for the viewer. The precision of the robotic camera along with the Abdel-Gawad's cinematography does make you feel like you're dancing ballet along side the actors. We can't even imagine the effect of equipping the robotic arm with a green-screened circular track that gave it 360 degrees of tracking capability. But we would love to see it. Check out the behind-the-scenes footage of how "Francesca Da Rimini" was made and let us know what you think about the practical and artistic potential of this technology.