Researchers at the Auckland University of Technology have developed a haptic glove which can provide tactile feedback and control videogames, for instance allowing a user to shoot guns with a thumb and missiles with fingers, and to control game environments with hand gestures.
The glove features a custom-fabricated printed circuit board which employs an Arduino micro-controller system, all grafted onto a custom-knitted glove (see image below). The glove utilises an Inertial Measurement Unit (IMU) facilitated by Micro-Electro-Mechanical Systems (MEMS) – the same kind of gyroscopes, accelerometer and magnetometer technology which makes smartphone adaptation of VR (i.e. Cardboard and Gear) practical.
The glove was prototyped in a few days, with mission objectives modified by practicality of available and affordable technologies. Initial work led to experiments with ‘muscle wire’ as a way to provide compressive force – apparent ‘resistance – to the wearer’s fingers, using 3D-printers to demonstrate possible designs for economical flex-sensors.
The sensor uses tubing which contains an infra-red sensor at one end and an IR receiver at the other, signalling changes in resistance as the tube is bent.
Later versions of the haptic glove were tested against a game using the Unity game engine. Events in the Unity program trigger vibrations in motors mounted on each of the fingertips of the glove, which can connect powered via cable or on battery via Bluetooth.
The Unity game is a basic flight simulator controlled by the haptic glove in combination with a Microsoft Kinect. The user can fire weapons in the game by bending their thumb or clenching a fist, with vibratory event feedback. The plane’s flight pitch, yaw and velocity are controlled by inclines of the hand and proximity to the monitor on which the game is played out. The user can return to the main menu by placing their right hand diagonally down.
The researchers describe the game as ‘rather limited’, wherein the player can shoot at enemy vessels whilst in flight, but receives no return fire. The paper’s authors, led by Jacques Footit, say of it: ‘Whilst perfectly functional as a demonstration of the haptic glove, the game would have little appeal as a game in its own right and would likely have limited ability to engage a player for more than a few minutes.’ Nonetheless the paper observes that a group of students who were invited to try out the haptic glove with the Unity game engaged with it ‘for much longer than would have been expected given the simple implementation.’
Haptic uptake and resistance
The team are interested in experimenting with force-feedback in future work on wearable haptics, as well as low-level electrical muscle stimulation, commenting ‘Whilst not strictly required for a game controller, the ability to provide force-feedback would allow the haptic glove concept to be extended to a broader range of virtual environments.’
Interest in haptic controllers is currently ramping up in response to developing markets in virtual and augmented reality. The University of Sussex is developing a ‘SkinHaptics’ system which effectively turns the palm into a touch-screen; the Institute of Electrical and Electronics Engineers is considering whether our networks are ready for the advent of the ‘haptic internet’; and Apple is patenting haptic technology – including temperature factors – for future devices.
It does seem, however, that dexterous interaction with virtual objects is inevitably going to be a rather ephemeral experience without the adjunct of real world objects. Pressure-inducing bladders can potentially form a virtual trigger on your finger, but can’t convey the weight of the gun. Resistance and real-world kinetics would be hard to achieve even with an unaffordable exo-skeleton capable of locking the user’s limbs to signify an object’s mass and resistance.
Additionally even a glove covered with motion capture sensors will need a real-world base against which to calculate relative movement; also the need to compute high volumes of MoCap data in real time means that the earliest VR/AR haptic controller systems seem likely to provide representational movement rather than acute and accurate responsiveness.
Interacting with photo-real virtual worlds which convince and yet have no resistive substance posits the possibility of some interesting new ‘augmented’ gaming injuries – Boy walks into wall after 14 hours playing Call of Duty: Off World.
The Auckland researchers suggest that haptic feedback is likely to find new and possible more valuable uses beyond attempts to simulate resistance, texture and topography:
‘Vibrotactile haptic feedback can be very useful for conveying information to a user that is not intrinsically haptic in nature. It also indicates that for many situations true and realistic haptic simulations are not necessary. Where true and realistic haptic feedback is not required, vibrotactile feedback provides an easily implemented, affordable and versatile solution’