When a movie is more CG than it is real â think movies like Avatar or any of Pixarâs stuff, where all or nearly all of the environment is rendered â a new challenge appears: the camera.
Getting a fake camera (like the one in a rendering program) to move and behave like a real camera (like the one that the camera guy is traditionally holding) can be a pain. Testing a scene from a whole different angle is less âhey, letâs try that again real quick and Iâll shoot it from over there,â and more âhey, letâs tear open keyframes and rework a bunch of carefully set parameters.â
James Cameron and the folks at Pixar have been solving this with âvirtual camerasâ â physical camera-like devices that let videographers shoot fully virtual scenes much like they would shoot any scene in the real world. They tie a simulated camera to the movements/orientation of a real-world camera proxy, and push everything the simulated camera âseesâ back to a display on its real-world counterpart in real time.
As you might expect, these virtual camera rigs areâ¦ not cheap.
So what are you looking at?
Kert is having Space Pirate Trainer render another view of the game to a second camera, independent of the playerâs, in one corner of his screen. Heâs using an open source project called jsmpeg-vnc to push what this camera sees to his iPhone in near real time. Meanwhile, heâs using a third HTC Vive controller (beyond the two the player uses for the game) to capture the location and orientation of the rig, and is using that to control the aforementioned in-game camera view. Strap the iPhone and an HTC Vive controller to a handheld stabilizer and bam! â heâs got a virtual camera built out of gear he already had lying around.
It doesnât look like it does all the crazy stuff the Cameron/Pixar rigs do â those can do stuff like tweaking the simulated F-stop and focus (which would have to be accounted for at a software level in every game/environment with a rig like this in mind, which complicates things a good bit). But for a project hacked together out of existing components, this is awesome â and hopefully, just the first step.