Researchers at Carnegie Mellon University's Robotics Institute have enabled a computer to understand the body poses and movements of multiple people from video in real time, even down to the details of the pose of each individual's fingers.
This new method was developed with the help of the Panoptic Studio, a two-story dome embedded with 500 video cameras. The insights that were gained from experiments in that facility are now making it possible to detect the pose of a group of people using a single camera and a laptop computer.
This research opens up new ways for people and machines to interact with each other, and for people to use machines to better understand the world around them. The ability to recognize hand poses, for instance, will make it possible for people to interact with computers in new and more natural ways, such as communicating with computers simply by pointing at things.
Detecting the nuances of nonverbal communication between individuals will allow robots to serve in social spaces, which will allow robots to perceive what people around them are doing, what moods they are in and whether they can be interrupted.
Self-driving cars could get an early warning that a pedestrian is about to step into the street by monitoring body language. Enabling machines to understand human behavior also could enable new approaches to behavioral diagnosis and rehabilitation for conditions such as autism, dyslexia and depression.
In sports analytics, real-time pose detection will make it possible for computers not only to track the position of each player on the field of play, as is now the case, but to also know what players are doing with their arms, legs and heads at each point in time. The methods can be used for live events or applied to existing videos.
To encourage more research and applications, the researchers have released their computer code for both multiperson and hand-pose estimation. It already is being widely used by research groups, and more than 20 commercial groups, including automotive companies, have expressed interest in licensing the technology.
When the Panoptic Studio was built a decade ago with support from the National Science Foundation, it was not clear what impact it would have. Now, researchers are able to break through a number of technical barriers!
Xilinx FPGA enables machine learning, computer vision, sensor fusion, and connectivity into vision guided intelligent systems
- Deep Learning, Computer Vision, and Sensor Fusion
This demonstration uses the three complex algorithms most often used in vision-guided systems today including Convolutional Neural Network for object detection or scene segmentation, Dense Optical Flow for motion tracking and Stereo Vision for depth perception, running on a single Xilinx Zynq Ultrascale MPSoC device.
EarthTron LLC is an Unlimited Franchised Distributor of Electronic Components and Products. We provide customers with services not available from a restricted Franchised Distributor. In addition to our franchised lines we give you the convenience of only having to go to one company for ALL of your electronics needs. Our experience in the industry gives you the confidence that we only source from quality suppliers, to deliver quality parts.