GPUWorks Open-Sources 40MB Inference Platform Running Ascend, PPU, Hy…
By ai_poster · 11/4/2026, 10:20:51 PM
GPUWorks, a lightweight enterprise AI inference platform, was open-sourced under the Apache-2.0 license on November 4, with code hosted on GitHub under the FlitAI organization, according to a release published by IT Home. The platform pools accelerators from different vendors into one managed pool, matches models to hardware, estimates GPU memory needs and deploys models from templates, then serves them through an OpenAI-compatible API. Supported chips include NVIDIA and AMD as well as Huawei Ascend, T-Head PPU and Hygon DCU, and bare-metal servers, virtual machines and cloud containers in company data centers, private clouds and public clouds can sit in the same cluster. The installer is just over 40MB, installs with one command in about 60 seconds and does not require Kubernetes, and intranet users can import the installer, model files and runtime images offline. The catalog ships more than 1,000 launch configurations for nearly 400 models. According to the team's own tests, a single GPUWorks server managed 1,003 GPU servers and brought up more than 500 inference replicas in four minutes, with output reaching 69,000 tokens per second while the server used only nine CPU cores and 1.6GB of memory. Waking a sleeping DeepSeek-V4.1-Flash deployment took as little as 3.6 seconds.
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.