Microsoft also claims that it can recognise emotions, and Natal has impressed game players, developers and movie mogul Steven Spielberg alike.
It is undeniably impressive. And you have to wonder: how does Natal actually work? Microsoft is remaining tight-lipped. But we've talked to industry insiders and pulled together some material from the New Scientist archive to suggest how it could live up to the high expectations generated by partially enhanced concept videos like this one.
Camera capture
Organic Motion is one of the few commercial companies currently offering markerless motion capture technology. Andrew Tschesnok, the firm's CEO, says his system uses as many as 14 cameras positioned around an actor to get a 3D picture of their movement in real time.
But Natal comes equipped with just two cameras. So how can it work with such a limited input?
In-depth view
A year ago, Israeli firm 3DV Systems unveiled a system that uses an infrared depth-sensing camera. The ZCam is said to be able to pinpoint the depth of an object to within 1 to 2 centimetres, and capture the information at a rate of 60 frames per minute for very smooth motion. A second, full colour camera in the device records textures and colours.
In March of this year, the company sold its assets to a third party, reportedly Microsoft. Many people have concluded that Natal's motion capture system owes a lot to the ZCam. When New Scientist asked 3DV System's CEO Zvika Klier about the possibility this week, he had no comment.
Tschesnok says the results from ZCam-like systems fall short of what he'd call true markerless motion capture – Natal can't see what's going on behind a gamer's back – but it is still an improvement over other games consoles' interfaces and will work well for Natal's needs. "In a way, it's the same effect as having a Wiimote in each hand, one on each foot and one on your head," he says.
But more is yet to come: Tschesnok predicts that the games consoles due out early in the next decade will use his multi-camera system, built into the widely used "surround sound" speaker systems to provide users with a truly immersive experience.
People are also understandably excited about Natal appearing to recognise and respond to voices and emotions. However, while speech recognition software may be increasingly common, it still suffers from the "cocktail party" effect; where background noise and multiple voices can cause errors.
Finding a way round this issue is vital for gaming. Microsoft is targeting Natal at social and casual gamers who are likely to play in the company of others, yet few, if any, voice recognition systems have been created capable of following a single voice in a noisy room.
IBM has shown that a system that can read lips provides one way around that. That could help Natal, but would need to be tuned separately to different languages.
Research that enabled one of Honda's Asimo humanoid's to understand three voices all speaking at once even suggests that multiplayer voice control could be possible one day.
Speech software could also feasibly be used to recognise emotions from stresses and emphasis in a player's voice. For example, software called Emotive Alert is able to classify voices into one of eight basic emotional states.
We are told, though, that Natal will use facial recognition, so it may read emotions that way too. Earlier this year, artist Tina Gonsalves teamed up with neuroscientist Chris Frith at University College London, UK, to develop an art installation that responds to the emotion of visitors, using an algorithm Frith created to read faces.
Frith told New Scientist that getting such systems to perform reliably is difficult, but not impossible. However, it is unlikely that Natal will detect more than a handful of basic emotions.
Even with that limitation, a console able to sense emotions and respond to them has the potential to make gaming a much richer experience.
In 2005, an "emotionally aware" virtual fitness trainer, name Laura, was tested on groups of volunteers. Her friendly gestures and sympathetic body language were found to genuinely foster a better connection with users. This positive influence significantly increased the participants' exercise levels compared to a control group interacting with a version of Laura that didn't recognise emotions.
The Natal concept videos may seem on first viewing to show technology bordering on the fictional, but it's clear that many of the basics are within our technological reach. How many of these features gamers will see when Natal finally hits shelves, though, is unknown.
Source: New Scientist
No comments:
Post a Comment