Real Actors in VR

We want to create an VR experience which is as realistic as possible, so players can immerse deeply in our world. One of the most important elements to achieve that, are our characters. However, creating believable human characters is one of the biggest challenges in computer animation. Artists quickly tend to portray people in a perfect way, but it is the blemishes and asymmetries of a person that make the appearance alive and thus the character realistic.

That’s why we decided to scan our actors in Quanero 2, using photogrammetry. This technique makes it possible to develop a three-dimensional model from photographs, by using many cameras, all pointed at the object you want to capture. A big advantage of this method is, that the proportions of the digital doubles fit exactly to those of the actors. There is no so-called “offset” in the motion capturing, a distance between the movements of the actors and their digital avatars, which makes the movements of the digital characters seem unnatural.

We also put a lot of attention to the faces, which we scanned separately, to achieve as much details as possible in the expressions. For that purpose we have built our own camera rig with 24 DSLR cameras, all focused only on the face of the actor. As a results, individual wrinkles and pores on the 3D model and in the textures are visible. But, one scan per actor is not enough to cover the facial expressions the actors can deliver. So we created over 2000 photos capturing different expressions per character to display their rich bandwidth of facial movement. The result is, that our realistic characters and their performance can be viewed freely from any direction in VR.

At this point we would like to thank the company VirtuMake from Vienna. They provided us their studio with over 60 DSLR cameras, for the full body scans of our actors.

Many thanks also to the developers of CapturingReality, who provided us with their photogrammetry software. The high degree of detail and the quality of our models would not have been possible without their software.

Wolfgang Stockinger · MoCap and Scan Supervisor