Baseten Opens Serverless Loops for Post-Training, Billing by Token

Baseten made Serverless Loops generally available on November 6, 2026, opening its managed SDK for supervised fine-tuning and reinforcement learning on open models, with per-token training billing and a direct path from checkpoints to Baseten Dedicated Inference. The company says its production inference stack already serves more than 15 billion requests daily.

Loops splits the work cleanly. Users supply the training signal. Baseten operates the GPUs, kernels, model architectures and distributed-training backend. The Python SDK trains LoRA adapters with base-model weights frozen. Users choose the base model, LoRA rank, learning rate, loss function, data or environment and algorithm; Baseten runs the trainer, sampler, weight synchronization and supporting infrastructure.

What does the serverless catalog include?

Three models ship in the serverless catalog: Qwen/Qwen3.6-35B-A3B and zai-org/GLM-5.3-Flash for text and vision, and zai-org/GLM-5.3 for text only. Each supports a maximum sequence length of 262,144 tokens. GLM-5.3's long-context configuration does not support cross_entropy or dpo losses. Dedicated mode covers the wider model catalog, bills for GPU time and adds controls for replicas, warm reuse and spot capacity.

Serverless runs use shared capacity without GPU reservations. Billing covers tokens used in training and sampling only. Time spent reading results, changing a reward function or sitting idle incurs no token charges. Default mode can fall back to dedicated capacity for an unsupported model, LoRA rank or sequence length, or when a run reuses another run's deployments. Setting the scope to serverless makes an unsupported request fail instead.

One caveat matters for planning. Model enablement does not guarantee provisioning. Workspace-specific access, unavailable GPUs and organization trainer limits can still block a run, giving the pay-per-token model practical capacity boundaries beyond the published catalog.

A compatibility package preserves Tinker imports and core training and sampling calls for teams migrating existing scripts. The migration guide still changes authentication to a Baseten API key, returns checkpoints as paginated file URLs, requires inline image inputs and fixes LoRA module coverage by model family.

How do checkpoints reach production?

The quickstart exposes four core operations:

• forward_backward for loss and gradient work
• optim_step for adapter updates
• sample for generation
• save_state for resumable training

Publishing sampler weights creates a deployable checkpoint and a sampling client pinned to it. A separate save_state operation preserves model and optimizer state. The first sampling request can take several minutes while serverless capacity starts — a practical consideration for short experiments.

A sampler checkpoint deploys as the base model with the trained LoRA adapter loaded. Checkpoints outlive their runs. Teams can deploy them through the CLI, REST API or dashboard using a Hugging Face access token, because the deployment downloads base weights. The resulting OpenAI-compatible endpoint bills for its GPU while live. Deleting that deployment stops the GPU charge without deleting the checkpoint; deactivating the Loops run does not remove the deployment.

For GA, Baseten improved handling of transient network errors and GPU failures, reduced startup overhead and accelerated weight transfers between trainers and samplers. Users can download weights in Hugging Face Safetensors format. The production path includes autoscaling, deployment management and inference metrics.

Who validated it, and with what results?

Baseten validated the service with numerical tests and end-to-end reinforcement-learning experiments using τ-bench tasks across retail, airline and banking, followed by evaluation on held-out hard tasks. The τ-bench project tests whether agents can converse with users, call tools, retrieve knowledge and follow policy across enterprise domains, with outcomes checked against verifiable database results.

The customer numbers stand out. Baseten reports that NVIDIA post-trained Nemotron Super 3.5 VL on text and vision tasks, improving intent-classification accuracy from 76.8% to 94% and reaching 87% validation F1 on receipt extraction; the work also included reinforcement learning for visual object counting.

Datadog is using multi-turn agentic RL to test training ideas for its SRE agent. Ambience Healthcare post-trained models to correct clinical transcription errors and capture medical terminology and formatting accurately.

LangChain Labs is post-training models on agent traces for LangSmith Engine, including work that reads a GitHub repository to diagnose an issue and drafts the prompt or code change for a pull request. It can switch to a smaller open-weight model for classifying traces by failure mode and severity or mapping them to existing issues.

The customer accounts document use before the GA announcement and show the intended loop: application data becomes training input, checkpoints are evaluated, and selected models move toward production. For infrastructure teams, the cost boundary is equally clear. Supported serverless runs avoid GPU reservations and idle-time token charges, while a live inference endpoint continues to bill for its GPU — and the serverless catalog plus workspace capacity determine where the serverless option applies at all.