Artificial intelligence (AI) is nothing new, but it is having its latest day in the sun because, at last, there is sufficient affordable compute and storage power to make it viable outside specialized labs. With associated disciplines like machine learning (ML), it has the potential to make sense of the vast quantities of data generated by connected people, cars and ‘things’, and to transmit instructions back to those objects so that they can act autonomously.
Does AI help MNOs, or limit the need for their services?
Those processes appear to present a significant opportunity for mobile operators, since the connected robots, cars and sensors will have to communicate with the central AI engine in the cloud, often over mobile links. But those links are themselves challenging, since many AI-enabled applications will require very high quality of service and very low latency. That has led to high interest in pushing as much AI processing as possible to the edge of the network, to reduce the distance to the engine, and even to the device itself.
That could reduce the need for ultra-low latency, one of the principal justifications for investment in 5G, and some AI platforms are adopting a form of ‘batch processing’, with a high percentage of activity going on at the edge and selected results uploaded to the central cloud periodically to enrich the machine learning base. This makes it important, for mobile operators, that they can monetize AI even when it is at the edge, whether by supporting shorter range links (from a gateway to a device perhaps, or between an autonomous car and roadside infrastructure).
The right balance between the economies of scale and efficiency of a centralized, cloud-based system, and the responsiveness of a distributed, edge-based approach, is one of the key decisions facing enterprises and service providers – and that, in turn, will affect how mobile operators need to plan their networks.
Elements of both will be required of course. Some AI-driven processes will remain too compute-intensive, security-sensitive or un-usual to be distributed out to smaller gateways and devices. But the balance of investment and enthusiasm currently seems to be tipped towards the edge, and there is a race to support AI/ML applications on small platforms, which is seeing a shifting definition of where the network edge actually lies.
Neural processing moves from supercomputers to devices
Qualcomm and Intel are battling for dominance here, as in so many other areas of the networked society and the Internet of Things. In the past month, Qualcomm has made its Snapdragon Neural Processing Engine software development kit available, providing programmers with tools to create on-device neural network-driven applications. And it has acquired Dutch start-up Scyfer, a spin-off from the University of Amsterdam, which has been developing a deep learning platform that has been used in several vertical markets such as manufacturing, healthcare and finance, and is also heavily edge-focused.
Qualcomm argues that carrying out most of the AI processing on the device can improve reliability, privacy protection and bandwidth efficiency, compared to solutions which have to transmit data to the cloud for processing.
“We started fundamental research a decade ago, and our current products now support many AI use cases from computer vision and natural language processing to malware detection on a variety of devices — such as smartphones and cars — and we are researching broader topics, such as AI for wireless connectivity, power management and photography,” said Matt Grob, EVP of technology at Qualcomm.
Meanwhile, Intel has shown off the latest design from Movidius, the machine vision processor startup it acquired last year to strengthen its IoT and AI strategies.
Movidius shows off Myriad X
The Myriad X vision processing unit (VPU) made its debut last week and claims to be the first system-on-chip with a dedicated Neural Compute Engine. It is “specifically designed to run deep neural networks at high speed and low power without compromising accuracy, enabling devices to see, understand and respond to their environments in real time,” said Intel.
The low power SoC is a successor to the Myriad 2 but makes a significant leap in functionality and performance. It can handle four trillion operations per second (up from a peak of 1.5 trillion) on an edge-based platform or device, enabling it to sense change in its environment and take action accordingly. It can support challenging tasks such as instant haptic feedback for remote surgery, or making inferences in gaming or decision support – rather than having to communicate with the central engine every time.
The SoC runs on 16 of Movidius homegrown cores (up from 12 in Myriad 2). It calls its DSP (digital signal processor) core architecture SHAVE (Streaming Hybrid Architecture Vector Engine). Now, on top of that, it has the new neural compute engine to provide localized deep learning capabilities. This engine is built on over 20 enhanced hardware accelerators which perform specific tasks optimally, and without adding to the compute overhead. Examples include depth mapping to help drones to land, or optical flow for very high performance motion estimation (for surveillance cameras that need to track large numbers of people or objects).
Intel Movidius exec Remi El-Ouazzane told EETimes: “Your archi-tecture needs to deal with new types of workloads in hardware microarchitecture, while DSPs are super-useful in running new types of computer vision and deep learning algorithms.”
The memory architecture, as well as the accelerators, helps to achieve the low power consumption which is critical to doing AI at the edge. Minimizing off-chip data transfer helps keep the power budget to 1W – on-chip memory is 2.5MB, from 2MB in Myriad 2.
“With this faster, more pervasive intelligence embedded directly into devices, the potential to make our world safer, more productive and more personal is limitless,” El-Ouazzane wrote in a blog post. The main target applications are currently in connected vehicles, as well as drones, robotics, smart surveillance and virtual reality headsets.
Read the source article at Rethink Technology Research.