Big Minds – Google Reboots Neural Networks
A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discriminatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognising highly overlapping digits.
Sabour, S, Frosst, N, & Hinton G E (2017). Dynamic Routing Between Capsules.
Google’s Geoff Hinton has unveiled a reboot of standard neural networks – ‘capsule networks’.
This technology is still at laboratory stage for now and has been reported in two papers (Here and Here) however the concept is fascinating. Capsule networks use neurons, grouped as capsules, arranged into layers which identify images or videos. When in agreement, capsules will confirm detection and escalate to a higher capsule group. This hierarchy ascends until the neural network makes a judgement about what it is seeing.
Hinton hopes that their capsule network will allow technology to correctly identify objects from all angles in substantially less time and using less compute resource.
Though still experimental, the potential for artificial intelligence object and facial recognition is vast.