Even as Nvidia thrives in the generative AI market with its extensive hardware and software offerings, startups like Hailo are emerging to compete. Hailo, an Israel-based startup, has introduced the Hailo-10, a new energy-efficient generative AI accelerator designed specifically for edge devices.
The Hailo-10 processor enables generative AI applications to operate on edge devices—such as vehicles and commercial robots—without reliance on cloud data centers. Hailo claims this will significantly enhance model performance while reducing costs and energy consumption.
In conjunction with the Hailo-10 launch, the company announced an extension of its Series C funding round, securing an additional $120 million. This investment, led by both new and existing investors across various industries, values Hailo at $1.2 billion. "This funding will allow us to capitalize on exciting opportunities in our pipeline and positions us for long-term growth," said Hailo co-founder and CEO Orr Danon. He emphasized that the Hailo-10 accelerator will integrate generative AI seamlessly into daily life, empowering users to leverage tools like chatbots and content generators with unparalleled flexibility.
What to Expect from the Hailo-10 Generative AI Accelerator
Generative AI is already transforming enterprises by creating marketing content and simulating human-like conversations. However, its edge capabilities remain largely untapped. Hailo aims to bring sophisticated functionalities—such as robots capable of natural human interaction—to reality with its new processor.
While traditional AI hardware in cloud data centers can support edge applications, cloud computing often suffers from latency due to data processing delays. The Hailo-10 overcomes this challenge by running generative AI services directly on devices, enhancing their processing capabilities.
Whether automating real-time translations, generating software code, or creating images and videos from text prompts, Hailo-10 allows users to execute these tasks directly on PCs or other edge systems without straining the CPU or draining the battery, according to Danon.
Using the same software suite as its predecessors, Hailo-8 and Hailo-15, the Hailo-10 operates at an impressive power-to-performance ratio. It can run Llama2-7B at up to 10 tokens per second using less than 5W of power, while generating images with Stable Diffusion 2.1 in under 5 seconds.
As edge generative AI becomes more immersive, the focus shifts to managing large language models (LLMs) within minimal power envelopes—ideally under five watts. The Hailo-10 achieves up to 40 TOPS (tera operations per second), setting a new performance benchmark for edge AI accelerators.
In comparison, Nvidia's Jetson line, which handles edge AI workloads, delivers up to 275 TOPS but also consumes more power. For entry-level edge AI applications, Nvidia's Orin Nano offers up to 40 TOPS with power usage ranging from 5W to 15W. Danon highlighted that the Hailo-10 demonstrates superior performance for inferencing over GPU-based solutions, thanks to its cost-effectiveness and reduced power needs, making it suitable for more compact edge systems.
Initial Target Industries for Hailo-10
Hailo plans to target the PC and automotive infotainment sectors when the Hailo-10 begins shipping in Q2 2024. However, it remains to be seen when it will expand to applications like generative AI-powered robots, an area already explored by Nvidia with its GR00T project.
Currently, Hailo partners with 300 global clients across various sectors, including computing, automotive, security, Industry 4.0, retail, and healthcare, featuring notable companies such as NEC, Bosch, Schneider Electric, Dell, ABB, and Foxconn.
In conclusion, Hailo’s innovative Hailo-10 generative AI accelerator could redefine edge computing, offering energy-efficient alternatives to existing solutions while enhancing the capabilities of a range of applications.
Ranking
