Event-Based Vision for Machines Inspired by the Human Visual Processing System

Innovation: Industry

Prophesee

2019

Image: Harry Quan / Unsplash / CC BY SA - Creative Commons Attribution + ShareAlike

Sense Light (Visible Spectrum) From the Environment

Living systems constantly receive signals from their environment that help them survive. Light (in the visible spectrum) can come from other living systems (such as fireflies) or from non-living sources (such as the sun). Survival often depends on sensing and responding to challenges like low light conditions or light that has been altered in some way. Because basic survival is at stake, living systems must excel at meeting those challenges. A well-known phenomenon is how water bends light. A stork trying to catch a fish underwater can compensate for this bending effect so that when it strikes at the fish, it has a good chance of catching it.

Search this Function

Compute

Computation allows an organism to take in and process information to make decisions. Whether it be calculating how fast prey is moving or deciding where to build a nest, computation is an important part of an organism’s life.

Search this Function

Encode/Decode

Organisms are constantly taking in new information, processing it, and storing that information as memories. Sensory information and stimuli such as sounds, smell, time, and spatial organization are encoded along with our memories. Whether we’re learning a new skill or repeating a task we’ve done before, we can easily call upon and decode that information when we need it.

Search this Function

Transduce/Convert Signals

Sometimes a signal, such as sound or vibration, reaches a living system and needs to be converted into a different, usable form. This requires converting from one type of energy form to another. For example, organisms with eyes have photoreceptors at the back of the eyes that convert light into electrical pulses. These pulses travel to the brain and allows it to register color, shape, and detail.

Search this Function

Brochures

Prophesee White Paper
5.1 MB PDF

Metavision from Prophesee is a machine vision system that uses event-based vision to capture and record movement.

Benefits

Increased efficiency
Continuous data collection
Reduced energy use
Operable in poor and variable lighting

Applications

Machine learning
Autonomous driving
IOT devices

UN Sustainable Development Goals Addressed

Goal 9: Industry Innovation & Infrastructure

The Challenge

The conventional video camera takes an arbitrary number of frames per second, regardless of what motion it is trying to capture. When displayed rapidly, the images create an illusion of continuous motion, but between each individual frame, the camera is blind. Additionally, the camera repeatedly records background objects, creating an excessive amount of irrelevant data.

Innovation Details

Metavision® is a sensor that utilizes event-based vision to capture a continuous stream of information. If the camera is looking at a static scene it will generate no events, but if there is a burst of action, the camera adapts automatically to capture it instantly, similarly to the human visual processing system. Each pixel is recorded independently and is only recorded when it senses a change or movement. This makes it easier and more cost effective to acquire and analyze very fast motion, even if it is interleaved with times or areas in which motion is absent.

Biological Model

The photoreceptors in our eyes only report back to the brain when they detect a change in some feature of the visual scene, such as its contrast or luminance. Evolutionarily, it is far more important for us to be able to concentrate on the movement of a predator within a scene than to take repeated, indiscriminate inventories of the scene’s every detail. This process allows human vision to collect all the information it needs, without wasting time and energy reprocessing images of the unchanging parts of the scene.

AskNature