Zhitong Finance App learned that on October 16, Paradigm Intelligence (06682) and Beijing Ruisi Zhilian Technology Co., Ltd. (“SmartSphere”) announced the official launch of the one-stop production-level AI deployment platform “ModelSphere” (ModelSphere). The platform is free and open source for enterprise users. It integrates computing power management, model deployment, and operation optimization into a unified platform to help enterprises lower deployment and operation thresholds, and more efficiently transform computing power into safe, stable AI services that can be used in actual business.
As large-scale models enter the large-scale commercial stage, enterprises generally face problems such as low resource utilization, fragmented chip ecology, high deployment thresholds, and difficulties in optimizing production environments in computing power coordination, model deployment, and efficiency optimization. Model Domain is committed to integrating these tasks into a unified deployment and management process, reducing repeated investment in underlying system adaptation, resource scheduling, and daily operation and maintenance, so that enterprises can embed various open source models into core business processes without having to deeply grasp complex tasks such as driver compilation and hardware adaptation, so that technical teams can focus more on application development and business requirements.
The model domain has four core advantages:
1. Simplify deployment and ensure the stable operation of model services: The model domain provides graphical operation panels and pre-configured deployment capabilities, reduces bias in environment configuration codes, helps developers quickly build inference services for large models, and simplifies different chip clusters. The platform has built-in enterprise-level capabilities such as automatic elastic scaling, PD separation and deployment, three-level KV cache, high-availability load balancing, and automatic service interruption to support stable operation of model services in production environments.
2. Continuous automated tuning to improve model operation efficiency: The platform's built-in AutoTune intelligent model scheduling strategy and adaptive load balance scheduling combined with hardware acceleration capabilities optimize the ability of the model to process requests and the initial response speed. The ultimate throughput efficiency of the system was greatly increased by about 285% compared to the benchmark, and the performance jumped 4 times the original, and the “full” release of computing power was achieved.
3. Manage multiple chips uniformly to reduce repetitive adaptations: The platform integrates the EngineX inference engine to support and perform deep tuning for various computing power chips. It can unify and mix schedule domestic computing resources such as Huawei Ascend, Tianshu Smart Chip, Moore Thread, and Haiguang DCU, so users don't need to rewrite the underlying logic code for different chips.
4. Continuously optimize and adapt to mainstream models and accelerate application deployment: The model domain team continues to follow up the iteration of the open source community and has adapted and optimized mainstream models such as DeepSEEK-v4/v4-Flash, Qwen, Kimi, MiniMax, GLM, etc., to support rapid deployment and performance tuning to the production environment after model release to help enterprises implement the new model in actual business.
The positioning of the model domain in the AI technology stack
At the technical level, the model domain cooperates with inference engines such as VLLM and SGlang: the inference engine is responsible for improving the computational efficiency of a single server, while the model domain is responsible for cluster-level management and optimization, including load- and cache-aware request routing, unified KV buffer pool, automated scaling based on service quality, automatic service interruption and continuous writing, automated performance tuning based on online data, operation monitoring, and service management to ensure the high performance and high availability of the system in production environments.
The model domain is fully open source, opening up engineering capabilities for large-scale model deployment and optimization to more enterprises and developers, while gathering industrial strength to continuously improve support for different chips and models, reduce hardware adaptation barriers, lower the application threshold for large models, help enterprises make good use of existing computing power, and efficiently release the commercial potential of heterogeneous computing power.
The modal domain uses the loose Apache 2.0 protocol and is online on GitHub: https://github.com/modelsphere/modelsphere