Sony Talks Key Lessons for Artists Working with Virtual Reality

Virtual Reality (VR) is a technology that in recent years has seen an explosion of interest in the video game industry. What once was something of science fiction has now become a reality as companies compete to be at the top of the VR experience, with Oculus Rift arguably being at the top, but as more companies enter this market including Sony and Valve the race is at a full sprint. At GDC 2015 Jason Hickey of Sony gave an interesting talk about the key lessons he learned as an artist working with VR on projects like The Deep and Street Luge. Jason began the talk with explaining what the VR experience is like, "You've got to understand presence in order to understand VR. What we call "technical presence" is the hardware and the rendering engine working together to fool your senses into believing you're actually there. So no longer are you using suspension of disbelief or your imagination to fill in the gaps like you would a story or film, you actually feel like you're there." Jason went on to talk about the importance of creating an engaging experience for the user, and making sure it's correct, because one bad VR experience can influence how they feel about the technology overall. "If you do VR right, it's super powerful and engaging, but on the other side if you do it wrong you can make the player feel really uncomfortable and you can disengage them. So if it's the first game they play on VR, you're basically denying everyone else in this room who is working on something else the opportunity to show that audience their product because if you put someone off in VR they're probably never going to pick it up again. So be very responsible and think about your audience." Now that we have VR technology the next step is to understand how to enhance the experience and make sure you're creating something that complements the technology, "You've got to work together as a team to stay in budget, and what I mean by budget is the target frame rate, and that can be 60fps, 90fps 100fps, whatever it is you decide to do you've got to stick with that all the way through. That means no spikes, no 57fps if you're aiming for 60fps, so stick to this target frame rate through the entire production."

Art Direction

During the talk Jason outlined some of the most important lessons that every artist who is working with VR should know, the first is team building. "If you're going to a build a team with new artists the first thing you'll want is artists who love design and understand it. This is going to help a lot with iteration time, and it also means they are going to support new designers in your effort to make innovative gameplay." Jason continued, "secondly make sure that they love doing technical things, for you artists out there you need to enjoy optimizing and enjoy getting those frame rate budgets down. In your interview stage with artists you really need to be informing them of all the technical things involved with VR, because you can't fool someone who really just wants to make pretty assets into working on a VR project, they really need to love the whole product more than the asset."

• Review in headset

When it comes to art direction on a VR project you must review all assets in the headset, Jason simply put it, "If you're an Art Director and you're not making time to review in the headset then you're a bad Art Director. This is especially important for the final sign off."

• UI must be in world

Another tip Jason gave to artists is the fact that UI should really be in the world, and not just plastered onto the player's face because they can't focus on it. "It doesn't have to be massively projected like Splinter Cell style, although that's really nice, but you just need to think about it ahead of time. In general think about UI early, it always seems to be an after thought for games, but it's irresponsible to do that. So get the UI team in right at the beginning of the design."

• Fit for purpose, and test early

The next tip Jason gave is thinking about the purpose of each asset and making sure to test early, "The level of detail required of the art is that in the first two meters it's really important to have a high level of detail, but after about five meters it gets much more difficult to see those details. If you test a level you'll know where to spend your money. You also don't want crazy high poly for your greybox, but you also don't want to be lazy and have simple pieces of geometry as placeholders. You need to have an accurate polycount with LODs."

• Thin geometry is bad!

Jason stated, "Aliasing is the worst thing ever in VR, it's really awful. You can't really focus on stuff properly, and it just makes it horrible to watch. So some ways to remedy this is to avoid thin geometry, and get rid of wires and things like that. Same with high contrast materials like black on white, or really shiny chrome." The worst thing to do is lean on the rendering team to fix aliasing issues.

• Accurate scale is vital

Jason also spoke about why accurate scale is so important to the VR experience, "You judge your perception of your own size in the scene based on your relative scale. If you get that wrong, when you're looking at an asset you'll feel smaller, and with just one oddly sized asset in the scene it can make the entire scene feel wrong." The same goes for the characters in the scene as well, "Let's say for example, you're in virtual reality and you look down and have a massive belt on, you know it's much bigger than a regular belt, because you know what size a belt is suppose to be, so then this makes you think your head in the game must be massive as well."

• Large shapes are more important than high frequency detail

As with most designs you'll always to start with the large shapes first, and focus on things like silhouette first rather than small details, "Do the primary shape, the silhouette, then get into the secondary details. Make sure you get to the secondary details of a model and then test it in game and see if you need to add any more." Small details in VR like scratches can actually be a bad thing and end up distracting the player.

•  Stylized can work better than photo realistic

Jason went onto explain why he feels stylized works better than photo realistic for VR, "The reason it works better for me for of all is because it is cheaper to render generally, and secondly even if it wasn't I would probably still go down this route because it catches light better on simple forms. So if you've got really clear volumes in 3D you can recognize the shape really easily and therefore it's more interesting."

• Accurate Materials

Jason also discussed the importance of accurate materials when working with VR, "I personally would use physically based materials, clearly this will depend on your art direction, but if you are doing something slightly stylized, physically based materials will save you loads of time. We use Substance Painter and Substance Designer which has saved so much time, with one caveat I would say, build your own library of materials. That's not because the materials are necessarily bad from Substance, but if you're an Art Director you want to really have control, so build them from the ground up." Virtual Reality is a new medium and there are big differences from a typical game. Not only does design have to think of VR differently but so does the art team involved with a project. When working with VR the game should be built from the ground up with VR in mind, and not just ported like a stereo 3D movie.