Basics of Hand Interactions in VR
When working with VR, the most common way of interacting with the world is through the players hands. Whether is throwing, climbing, or shooting, we use the hands positional data along with along with inputs via the triggers and buttons to let the player interact with the game. But how do we go about setting that up? What does a hand interaction look like in its most basic sense? This article will go over just that, while keeping away from focusing on a specific engine, framework or integration.
The biggest question I get when it comes building mechanics in VR is where do I start? I like to break hand interactions into 3 states: On grab, While Grabbed, and On Release. Most frameworks have their own grab function built in, and you will see they follow this exact pattern, sometimes worded slightly different, but it follows the basic idea. But before we deal with our states we need to first take a look at grab detection.
Grab Detection:
Logic that determines whether an object can be grabbed. This is most commonly implemented by checking if the players hand is inside the same area as the object via triggers, or sending a message via raycasting. When the hand or ray enters the space we can directly interact with the object.
Okay onto the states.
Trending AR VR Articles:
1. The Possibility of AR on the Urban Space
On Grab:
This is the logic that executes when the object is picked up, it could be snapping an object to your hands position, changing its color, or playing a sound effect. It is usually started by the player pressing down the hand trigger.
While Grabbed:
This logic executes every frame that the object is grabbed. We can use this to check input, apply motion, or track our hands position. We can determine we are in the While Grabbed State by checking if the hand trigger is still down.
On Release:
This logic runs when we let go of an object, it could be used for adding force to a throw, restoring physics to an object, or snapping it back to a specific position. This is usually triggered, by the player letting go of the hand trigger, or the controller getting too far a way from the object you are interacting with.
Lets take a look at this in action.
1. The Drawer
On Grab:
Spawn a fake hand on the drawer and hide the players actual hand.
While Grabbed:
Update the drawers position to move towards the players controller.
On Release:
Hide the fake hand, and Reenable the players hand.
2. The Gun
On Grab:
Snap the gun to the players hand. Hide the players hand.
While Grabbed:
Update the guns position to track the hand.
Check for input of the index trigger. When pressed, play animation, particle effects and sound effect.
On Release:
Show the players hand.
Stop checking for input.
Let physics control the guns movement again.
3. Climbing
On Grab:
Hide the players hand, disable the players movement script(this will keep the player from being able to move via the thumbsticks and keep gravity from applying).
While Grabbed:
Track the players controller velocity, apply inverse movement to the players character controller.
On Release:
Show the players hand.
Reenable the movement script.
Breaking the logic down into these 3 events is a simple way to think about hand interactions without overwhelming yourself. Inputs you may want to check for during While Grabbed are:
Controller rotation — for turning knobs and rotating objects.
Button or trigger input — for interactions such as shooting
Controller Velocity — to see how fast the controller is moving, useful for throwing objects or moving the player character.
Controller Position — for having your interactable move or rotate towards your controller.
As an exercise break down the following interactions using On Grab, While Grabbed, and On Release:
Fire Extinguisher
Baseball
Air Horn
Fire Extinguisher:
On Grab: snap to players hand position, hide hand.
While Grabbed: track and follow hands position, detect index trigger for input, if pressed play particle effect.
On Release: Add velocity of hand to the object, allow physics to control its movement.
Baseball
On Grab: Snap to players hand position, hide hand
While Grabbed: Track and follow the hand position
On Release: Add velocity of hand to the object, optionally add a force multiplier to throw the ball faster. allow physics to control its movement
Air Horn
On Grab: Snap to players hand position, hide hand
While Grabbed: track and follow hands position, detect index trigger for input, if pressed play sound effect.
On Release: Add velocity of hand to the object, allow physics to control its movement.
Obviously your answers can slightly vary as there are a lot of different ways to accomplish these interactions but I hope breaking it down into these 3 events is an easy way of approaching hand interactables. VR can be a daunting medium to get into but if you approach it like regular game design all that really differs is how we handle input. As usual when you break a large problem down into smaller parts it becomes much more manageable.
Don’t forget to give us your 👏 !
