A Radically New Approach to VR
BY Karl Daum — October 18, 2021

Brett Jackson has such strong opinions about what makes a good VR experience that he has taken a year off to devote himself entirely to the production of a new game for the Oculus Rift and HTC Vive. It’s called Dimensional and it has entered its last day on Kickstarter, already fully funded.

In Dimensional, players navigate booby traps and hazards of the underground chambers built to protect a trans-dimensional alien race. The mission is to rescue their offspring before their hunters, who have drilled into these chambers, get there first.

rotatingPlatforms

A particularly treacherous hallway. 

For Jackson, VR should be all about immersion, a banality it would seem, given the technology. But with the inherent restrictions of VR headsets such as the Oculus or Samsung Gear, which don’t have the controller and spatial-mapping capacity of the HTC Vive, VR is mostly reduced to 360 screening. The best of these experiences bring a viewer through spaces visually like a moving camera. The less exciting have the viewer watching 360-degree TV. What we’re ultimately left with are surrounding experiences, but not necessarily immersive ones.

Dimensional radically breaks this mold by bringing the viewer into the game both spatially, and sonically. The major development in this regard is Dimensional’s unique grid system. By moving the headset around, players can map out whatever space is available to them in real life, which is in turn rendered as a grid in the game that the player can move around in.

Adding in height and size mapping, Dimensional makes the player feel truly transported into the experience. He or she can duck, peer, and move around in the grid to overcome obstacles in the map. To actually move through the map, it’s as simple as pointing and clicking to transport your grid, and yourself, to a new section of the level—as though you were a chess piece moving with your square.

Hoping to include breathing and dynamic sound registry, as well as rendered corporeal elements, Jackson aims to create a game that responds to the player’s presence—an interactive world as much as an immersive one. To bring the self into an experience is a massive step from being surrounded by one, and should Dimensional be successful, we might be witnessing the future of VR.

I recently interviewed Jackson about his game and the technology and techniques behind it. In Dimensional, we find an approach to storytelling that focuses on the player’s role in the experience that goes beyond the visual. By creating a malleable and responsive environment, Jackson generates a story that feels more lived than viewed.

 

This interview has been edited.

 

Karl Daum: What was your inspiration behind Dimensional? Had you played many VR games/viewed many VR experiences prior to taking on this challenge?

Brett Jackson: I’ve played quite a few tech demos, but my inspiration came from playing the Oculus Tuscany demo that comes with the DK2. This demo simply puts you in a location. One of the first things I did was arrange the tracking camera so that I could take a step and lean over the balcony. That simple action made the scene feel so much more immersive. It was too important to ignore.

KD: Outside of the Vive, VR headsets offer pretty static experiences, and many people often need to be reminded that the experience is on a 360-degree plane. This ultimately limits the VR experience to an immersive screening. But Dimensional breaks with these constraints with a pretty ingenious solution: the grid mapping. Talk about this feature. How did you arrive at the grid solution to the problems of spatial interaction and movement?

BJ: The locomotion system and floor grid were created as a response to my own physical space restrictions and the limitations of the hardware, together with my desire to physically move through the virtual world. Using the HMD to map out the grid on the floor and then teleporting that grid around the world seems like a fairly obvious mechanism. I’m surprised more games haven’t utilized the idea, especially as I shared an early demo (The Play Pit) and detailed videos back in July 2015. I found that an additional benefit of an irregular-shaped floor grid was that it enabled you to understand your orientation in the real world without breaking immersion. This knowledge makes you feel more secure and relaxed. I included an indicator of the tracking camera’s location to strengthen the effect. The system quickly evolved from there. Action, character dialogue, and teleporting techniques are now regularly used to encourage the player to face the camera. This increases tracking efficiency and also stops you getting tangled up in cables.

KD: In your Kickstarter video, you assert, “I have strong opinions about what makes a good VR experience.” What makes a good VR experience? And what have been the shortcomings of VR experiences so far?

eggGrab2

Players must rescue the alien offspring. One of their unhatched eggs can be seen here, under threat by the intruding hunters.

 

BJ: VR is all about immersion, I want to feel as though I’m in a real space. I expect to be able to physically move around and be seen and heard. I should be able to influence the environment. The world should work in a familiar and expected way and react to my presence. I want the freedom to explore and closely examine my environment. On top of that, I keep in mind that emotion encourages immersion.

A lot of VR games are taking traditional games and reimagining them in VR. Bigger-budget games need to take a fairly safe route for financial security, so they may look towards existing popular genres. They’re creating great experiences, but I want to be more experimental, explore what can be achieved with the current hardware and find ways to overcome its limitations.

KD: How do you see Dimensional taking on the challenge of becoming a great VR experience that fully engages the VR headset?

BJ: I describe Dimensional as an experimental game. Being a solo developer gives me the freedom to try new ideas, quickly change direction, and push out more experimental gameplay. I want the game to continue to evolve, even after its initial release. I’ll include new technology as I get access to it (tracked hand controllers being the first) and make the most of the available hardware. Once I’m happy with the core experience, I want to concentrate on more subtle techniques that may not be so obvious but will still increase immersion, such as eye contact with game characters. I find that players don’t react strongly to positioned sound and have ideas to help with that. There’s so much more I’d love to explore.

KD: You focus heavily on how your experience won’t make the audience feel nauseated. Do you see nausea as a major obstacle for VR? How did you eliminate the nausea effect of VR in your experience?

BJ: I don’t think nausea will be an obstacle for the early adopters, but I think it will affect mainstream adoption. Hardcore gamers will not be put off, but we can’t expect all users to put up with nausea and play often enough to adapt. We know that using natural movement is fine, and controller-based locomotion is a problem for many. Dimensional aims to be extremely comfortable to play. Moving platforms stop when you teleport to them, and falling is even represented without moving the player. However, I’m allowing optional controller-based locomotion and the ability to enable realistic falling so the hardcore players can enjoy the experience too.

KD: What kind of responses are you hoping to elicit from your audience in Dimensional?

BJ: Personally, horror or violence doesn’t interest me. These induce fear through graphic representations and the feeling of a loss of control. Dimensional also plays with the feeling of fear and trying to give you a subconscious urge to move out of harm’s way. Heights and enclosed spaces will also make you feel uneasy. The difference is that you are always in control. You may hesitate, but then when you’re ready you’ll choose to overcome your fears. Then you’ll have a feeling of accomplishment. I also want to use humor and to build a connection with characters.

KD: To the effects of the previous questions, you talk in your Kickstarter about engaging the subconscious spatially with claustrophobia and fear of heights/falling. You also want to incorporate breath tech and sound detection to really heighten immersion. Could you elaborate on how you hope to incorporate these features? Does this kind of engagement create a more immersive experience in your opinion?

BJ: We’re designed to instinctively react to danger. Those instincts are easily triggered in VR and fun to play with. You don’t need lifelike graphics, and the immersion is undeniable. In an early test, I had players take a big step off the edge of a platform and told them another platform would swing up and catch them before they fell. Everyone hesitated, some for long periods. They knew they weren’t in danger but still held back.

I believe that the player will feel increasingly immersed as you accurately represent more of them in the virtual world. That includes seeing their hands, their breath, and seeing their voice influence the world. The HMD’s microphone seems to be overlooked for gameplay at the moment. I don’t think breath detection will be possible for now due to the position of the mic, but we can certainly pick up on the player’s voice. If you’re really present in the virtual world, the characters should be able to see and hear you. Even if they can’t understand your language, they should respond differently to a quiet calm voice and an aggressive loud voice. The mic can be used to interact with characters, to attract or distract them, or to scare them out of harm’s way. There’s lots more I would like to experiment with, but it seems worthy of investing some time and effort.

KD: Talk a little about the story in the game. All of these techniques are wonderfully inspiring when it comes to immersion, but you also hope to emphasize the story, which features a trans-dimensional alien race and a sassy robot. Talk about how you came up with this story. How do you see it unfolding? In what ways do you hope the game’s mechanics can interact with the narrative to create a more immersive experience, not just sensorially but also emotionally? 

Digby

Digby, the peeved robot.

BJ: The game evolved from experimenting with how I can make the most of the hardware to fully immerse the player in another world. Digby (the peeved droid who doesn’t appreciate being used as a projectile) exists because I needed to use him for interaction. The dangerous chambers exist because I’m playing with fear and subconscious responses. These give me the basis on which the story has to be built. The player would enjoy completing the challenges without a story, but I know a good story will keep the player engaged and drive them to complete the whole game. I don’t want to tell the story—there won’t be cutscenes. I want the player to find the story and then seek out the truth because they want to know more. They can choose to blindly follow their missions or become more engaged with the narrative and maybe approach things differently as a consequence. Players will need to use the locomotion system and physical movement (leaning over, under objects) together with interaction with physics-enabled objects to explore the environment and uncover the story.

There are three sides to the story: the trans-dimensional alien race being hunted to extinction, the hunters, and the Earthlings that chose to get involved. I want to avoid good vs evil and instead explore perspective and seemingly justified motivation from all sides. I hope to make the player empathize with each of the three sides. I also want to increase the player’s bond with Digby. Digby will make direct eye contact with you, he’ll react to your voice and look towards your mouth when you speak, and he’ll back off or look away when you shout at him. These simple reactions should help to make him a more believable character.