Red Hat Acquires Neural Magic to Boost Hybrid Cloud AI Performance
Red Hat, an open-source solutions provider, has announced its acquisition of Neural Magic, a Massachusetts-based company specializing in software and algorithms to accelerate generative AI (gen AI) inference workloads. The acquisition aims to make high-performance AI accessible across hybrid cloud environments, addressing key challenges in deploying large language models (LLMs), which typically require costly and resource-intensive infrastructure.
"Neural Magic's expertise in inference performance engineering and commitment to open source aligns with Red Hat’s vision of high-performing AI workloads that directly map to customer-specific use cases and data, anywhere and everywhere across the hybrid cloud," Red Hat said in a statement this week.
Addressing the Challenges
Red Hat intends to address the challenges of building cost-efficient and reliable LLM services which requires significant computing power, energy resources, and specialized operational skills by making gen AI more accessible to more organizations through the open-source vLLM.
Developed by UC Berkeley, vLLM is a community-driven open-source project for open model serving (how gen AI models infer and solve problems), with support for all key model families, advanced inference acceleration research, and diverse hardware backends, including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, Nvidia GPUs, and x86 CPUs.
Red Hat says Neural Magic's expertise in the vLLM project, combined with Red Hat's portfolio of hybrid cloud AI technologies, will offer organizations an open pathway to building AI strategies that meet their unique needs.
Democratizing AI
Founded in 2018 as a spinout from MIT, Neural Magic has developed technologies for optimizing AI models, particularly in model inference performance. Red Hat intends to leverage this expertise in the open-source vLLM project to democratize AI by offering scalable, cost-efficient options for businesses of all sizes, supporting various model types and hardware backends.
"AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized, and open platforms and tools a necessity, as they enable organizations to select the environments, resources, and architectures that best align with their unique operational and data needs," said Matt Hicks, President, and CEO of Red Hat.
Enhancing AI Portfolio
With Neural Magic's capabilities, Red Hat plans to enhance its AI portfolio, including Red Hat Enterprise Linux AI (RHEL AI) for developing foundation models, Red Hat OpenShift AI for machine learning across Kubernetes environments, and InstructLab, a collaborative project with IBM to advance open-source models. This expanded portfolio will enable enterprises to fine-tune and deploy AI models with flexibility and security across corporate data centers, cloud platforms, and edge locations, Red Hat said.
"Open source has proven time and again to drive innovation through the power of community collaboration. At Neural Magic, we’ve assembled some of the industry's top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities," added Brian Stevens, CEO, Neural Magic.
"As our clients look to scale AI across their hybrid environments – virtualized, cloud-native LLMs built on open foundations will become the industry standard. Red Hat's leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic's offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise," said Dario Gil, IBM senior vice president and director of Research.
The transaction is subject to applicable regulatory reviews and other customary closing conditions.
