AI on the Edge Evolving Rapidly with Specialized Chips

1979
The Intel Movidius Vision Processing Unit (VPU) is available within a USB that can plug into a range of devices.

By AI Trends Staff

Edge computing brings computation and data storage closer to where it is needed, to improve response times and save bandwidth. Now more AI is being incorporated into edge devices, from IoT devices to smartphones to automobiles, as edge compute power increases and AI algorithms improve.

As the edge computer processes, only a subset of data generated by sensors is sent to the cloud, saving on bandwidth and cloud storage costs, according to a recent account in Forbes.

Decisions can be made faster using AI models on the edge that have been trained via machine learning in the cloud. A deep learning model deployed at the edge may see slower inferencing. The GPU-powered performance of the cloud is not available to them. To bridge the gap, chip manufacturers are building accelerators that speed up the model inferencing on the edge. The processors take over the more complex calculations needed to run the deep learning models. This can speed prediction and classification of data ingested by the edge layer.

Three AI accelerators are mentioned: NVIDIA Jetson, Intel Movidius and Myriad chips, and Google Edge TPU.

NVIDIA Jetson is built for the edge; programming is compatible with the enterprise counterparts, but the GPUs draw less power than GPUs powering servers. The most recent addition is Jetson Nano, which comes with a 128 core GPU. Resembling the Raspberry Pi, the development kid enables hobbyists and professionals to build AI and IoT solutions. The software stack is called JetPack, and it comes with drivers and libraries that can run machine learning and AI models at the edge. TensorFlow and PyTorch models can be converted to TensorRT, a format optimized for accuracy and speed.

Intel Movidius and Myriad chips come from Intel’s acquisition of Movidius in 2016. The niche chipmaker built computer vision processors used in drones and virtual reality devices. The flagship product was Myriad, built for processing images and video streams. It is described as a Vision Processing Unit (VPU), for its ability to process computer vision. Packaged within the Neural Compute Stick, the chip can work with both x86 and ARM devices. The software platform is built to optimize the chip’s machine learning models. It can plug into a Raspberry Pi for running inferencing; the chips are available on USB sticks or add-on cards that can attach to PCs, servers and edge devices.

Google Edge Tensor Processing Units (TPUs) accelerate machine learning workloads in the cloud. Customers using Google Cloud Platform can connect to the Cloud TPUs to balance processor speeds, memory and high-performance storage resources. Edge TPU, designed to run at the edge, was more recently announced to complement the Cloud TPU. It enables inferencing of trained models to be performed at the edge. Perceived uses include predictive maintenance, machine vision and voice recognition. Google Edge TPU cannot run models other than TensorFlow at the edge, unlike NVIDIA and Intel edge platforms.

Device Maker Samsung Also Eyeing the Edge

Electronics giant Samsung is also eyeing AI computing on the edge. In a recent interview published in Samsung Next, the managing director of Samsung NEXT Ventures, Brendon Kim, outlined his vision for edge computing.

Brendon Kim, managing director, Samsung NEXT Ventures

“AI is going to be embedded in everything that we do, everything that we touch, and everything that we use,” Kim said. “A lot of what needs to be done is in edge computing investments,” according to  Kim.

Samsung’s plans include adding 1,000 scientists at AI-dedicated research centers by 2022, as part of a $22B investment in advanced technology.

Samsung is researching AI that learns from observing human trainers instead of from millions of samples in a database, leveraging advances in the fields of reinforcement learning and imitation learning. As part of this effort, Samsung has funded a startup, Coveriant. This spinoff from UC Berkeley is teaching industrial robots conceptual tasks that can be applied in a range of situations. The current practice is to program actions that AI-driven robots can perform the same way every time.

Kim envisions edge processors optimized for AI to get a boost from orchestration, the coordination of separate processors for a purpose. These could be mini-clouds that could combine the benefits of computing on the edge with the power of computing in the cloud.

Samsung sells 500 million devices a year, Kim said, and the company is committed to making them intelligent.

Read the source articles in Forbes and in Samsung Next.