NVIDIA EGX Pushes AI Closer to the Edge, Promises Faster Decision Making on Realtime Streaming Data
While NVIDIA’s graphics cards, Studio laptops, and Super rumors comprised the majority of green team Computex coverage, the company also took the wraps off EGX which could have a much bigger impact on consumers’ daily lives than those consumer-facing NVIDIA products, albeit indirectly. EGX is NVIDIA’s initiative to push AI (accelerated by GPUs and CUDA APIs of course) and more specifically to bring the computing power and software running the AI algorithms as close to the “edge,” or end users and data sources and devices, as possible while balancing on-premises costs, capabilities, and latencies against cloud-hosted strategies. NVIDIA EGX edge servers are aimed at healthcare, retail, manufacturing, transportation, and telecommunications industries processing and making decisions based on real-time streaming data. NVIDIA notes that by 2025 more than 150 billion sensors and Internet of Things (IoT) devices will stream data that needs to be processed by software that can perceive, understand, and act on the data.
EGX solutions span from small form factor single board computers like the Jetson Nano running between 5 and 10 watts doing ½ TOPS (e.g. running image recognition) all the way up to a full rack of NVT4 servers capable of cranking out 10,000 TOPS and performing real-time tasks like speech recognition.
On the software side of things, NVIDIA has partnered with Red Hat to integrate and optimize NVIDIA Edge Stack with the Open Shift Kubernetes container. EGX includes NVIDIA drivers, CUDA Kubernetes plugin, CUDA container runtime, CUDA-X libraries, containerized AI frameworks and applications (e.g. TensorRT, TensorRT Inference Server, DeepStream). The software is intended to be run on NVIDIA certified EGX servers which can download software from NVIDIA’s NGC registry.
NVIDIA further partnered with Mellanox (who they now own) and Cisco for storage and networking IP. EGX is suitable for on premises, hybrid cloud, and multi-cloud IoT configurations (depending on what sorts of things the platform is being used for) and is compatible with AWS Greengrass and Azure IoT Edge.
My first thought was the possible impact on the retail space and what more powerful edge hardware running AI software would enable, especially with regards to real-time identification, tracking, and advertising to customers as well as various asset protection and customer service applications. Companies are already slowly moving in this direction with retailers moving towards digital price signs, individualized pricing, image and facial recognition at self-checkouts, and even things like Amazon Go that offers a worker-less shopping experience from the customer’s perspective.
Interestingly, NVIDIA included examples from its partners, and they have gotten rather creative with the technology as is, much less what will be possible in the future. BMW is using EGX to optimize logistics while Foxconn has incorporated the technology into its production lines to perform automated quality assurance on products. Meanwhile GE Healthcare has allegedly been able to achieve faster MR acquisition times while increasing image quality and reducing variability. Finally, Seagate is reportedly using GPU vision to inspect quality of read/write heads in manufacturing leading to a 10% production line throughput increase and 300% boost to ROI efficiency and quality.
NVIDIA EGX Server and Micro-server partners include: Abaco Systems, Acer, Advantech, ASRock Rack, Asus, Atos, AverMedia, Cisco, Cloudian, Connect Tech, Curtiss-Wright, Dell EMC, Fujitsu, Gigabyte, HPE, Inspur, Leetop, Lenovo, Miivii, Musashi, QCT, Sugon, Supermicro, Tyan, WiBase, and Wiwynn.
The medical imaging stuff in particular is neat and the times I was able to attend GTC conferences I was able to see glimpses of what is and will be possible with GPGPU acceleration and better AI algorithms to produce better images to assist healthcare professionals and patients with more reliable results and improved early detection.
AI is still in its infancy and very much a buzzword today, but it is constantly evolving and increasingly becoming a part of our daily lives. I hope that the technology is used responsibly and can reduce barriers of independence for those with disabilities and not just to cram more ads into our faces. In the meantime, I’ll keep waiting for the promise of self-driving cars!
What are your thoughts on AI and NVIDIA’s EGX platform?
