ROSA: A Robotics Foundation Model Serving System for Robot Factories

0:00 Robot serving isn't an edge problem.

0:02 Robotics foundation models are making general-purpose factory robots

0:05 practical, but the systems that serve them are

0:08 stuck in an edge-computing mindset. They typically assume

0:11 one robot and one model on a dedicated

0:14 nearby GPU, and they typically optimize only to

0:17 minimize the latency of a single action model.

0:20 Real robotic execution needs multi-model pipelines, action generation,

0:23 planning, safety checking, progress monitoring, and the objective

0:26 that matters is factory productivity, not per-request latency.

0:37 The authors propose ROSA, a serving architecture built

0:40 on three principles: a shared GPU pool that

0:43 a whole fleet accesses over the network, a

0:46 robotics-aware declarative interface for per-task pipelines and per-component

0:49 service objectives, and factory-objective-driven scheduling.

0:57 The result: ROSA improves SLO-qualified factory productivity by

1:00 up to twelve times over conventional dedicated serving,

1:03 and by more than two times over shared-server

1:06 baselines.

1:10 Serve the fleet, not one robot.

Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimizing the latency of a single action model. In this paper, we propose , the Robotics Oriented Serving Architecture, an RFM serving system for robot factories designed around three key principles. First, adopts shared GPU-pool serving, allowing a fleet of robots to access powerful server-class GPUs over the network in order to improve inference performance, battery duration, and GPU utilization. Second, provides a robotics-aware programming abstraction and system design that supports multi-model pipelines, per-task performance requirements, and failure handling. Third, uses factory-objective-driven scheduling to maximize SLO-qualified factory productivity rather than minimizing individual request latency. We implement on top of Ray Serve for distributed orchestration, with vLLM, PyTorch, and JAX as model-serving backends, and evaluate it on both real robots and synthetic large-scale workloads. The results show that improves factory productivity by up to 12.06 over conventional dedicated serving systems.