OpenTSLM’s SageMaker Launch Reaches 45.9% on ARFBench
AWS says fine-tuning raised OpenTSLM’s ARFBench accuracy from 5.0% to 45.9%. The 5.24-billion-parameter model is now deployable through SageMaker JumpStart.
By Nathan Brooks
5 min read

What's News
- OpenTSLM-Flamingo Gemma 3 4B became available on Amazon SageMaker JumpStart on November 17, 2026.
- Fine-tuning raised accuracy on 135 held-out ARFBench questions from 5.0% to 45.9%.
- The model’s bfloat16 weights occupy about 10.5 GB of GPU memory.
- The default ml.g6.2xlarge endpoint has one 24 GB NVIDIA L4 GPU and 32 GB of host memory.
- Datadog’s ARFBench contains 750 question-answer pairs drawn from 142 time series and 63 internal incidents.
By Nathan Brooks
Fine-tuning lifted OpenTSLM’s ARFBench accuracy from 5.0% to 45.9% on 135 held-out incident questions, according to Amazon Web Services’ November 17, 2026 announcement. AWS made the 5.24-billion-parameter OpenTSLM-Flamingo Gemma 3 4B available on Amazon SageMaker JumpStart, its machine-learning model hub. Customers can deploy it to a real-time SageMaker AI inference endpoint within their own AWS environments.
OpenTSLM combines numerical measurements with text, including maintenance records, market news and engineers’ observations, and returns natural-language explanations and answers, AWS said. The JumpStart package pairs Gemma 3 4B with a time-series encoder and cross-attention layers so the language model can attend to encoded signals.
The OpenTSLM research paper separates Flamingo from SoftPrompt. Both treat time series as a native input modality.
SoftPrompt adds learned time-series tokens to the language model’s sequence; Flamingo uses a Perceiver Resampler to generate a fixed-size latent representation and gated cross-attention to integrate it. AWS said the JumpStart release makes “a native time-series architecture easier to deploy and adapt,” while its evidence remains tied to “particular windows, tasks and answer formats.”
The model and its research base
- Scale and memory: AWS reports that the model’s weights occupy about 10.5 GB of GPU memory when stored in bfloat16.
- Research base: The paper’s original evaluation covered ECG question answering, sleep staging and human activity recognition. AWS said that gave the managed release a research foundation rooted in medical and wearable data.
- Open development: The separate OpenTSLM repository is MIT-licensed and provides fine-tuning code, according to AWS. The project says pretrained checkpoints are available through its Hugging Face organization and identifies a collaboration between Stanford University’s Schmiedmayer Lab and ETH Zurich’s Agentic Systems Lab.
Endpoint limits and input windows
The default ml.g6.2xlarge endpoint meets the single-GPU, bfloat16 requirement, according to AWS. It carries one 24 GB NVIDIA L4 GPU and 32 GB of host memory.
- Request size: AWS allows up to four question instances per request. Each can contain one to 32 numerical series, with as many as 32,768 values per series.
- Normalization: The endpoint standardizes each series to a mean of zero and a standard deviation of one. AWS supplies the original mean and standard deviation as text so the model receives both shape and scale.
- Runtime: Real-time calls have a 60-second limit and a 6 MB request-body cap. AWS reports roughly 20 generated tokens per second on the default instance; instances run sequentially, and AWS recommends asynchronous inference for longer work.
- Terms: Customers must accept the Gemma Terms of Use to deploy the gated JumpStart model, AWS said.
Nexthink’s proactive IT workflow
AWS described Nexthink and Aionic Labs as building toward proactive help through Spark, Nexthink’s personal IT agent, so employees can receive support before asking for it. The workflow combines recent CPU, memory, disk, application-stability and connectivity measurements with timestamped events, device configuration and software versions, AWS said.
AWS said the model supplies a state summary with supporting signals, a recommended check and action, and an expected outcome that Spark can verify. In its low-disk-space example, the model links falling free space, a rising disk queue and more frequent application hangs to slow performance or difficulty saving files.
Spark checks whether an IT-approved cleanup is appropriate, shows the employee what would be removed and seeks confirmation before acting, AWS said. It then checks the result and offers escalation to IT support if the problem persists.
Fine-tuning for incident questions
AWS’s adaptation workflow uses TimeNet, which provides a common dataset format and Python API with connectors and immutable versions. TimeNet’s documentation limits it to data; training, inference, model definitions and evaluation metrics sit outside its scope.
AWS’s example loads a pretrained checkpoint from Hugging Face and ARFBench data through TimeNet, runs a SageMaker training job, stores the adapted model in Amazon S3 and deploys it to a separate real-time endpoint with the same request format. Datadog’s ARFBench contains 750 multiple-choice question-answer pairs derived from 142 time series across 63 real internal incidents, using production telemetry and additional context.
The training job updates the time-series encoder, Perceiver Resampler, cross-attention layers and language-model token embeddings while keeping the rest of Gemma 3 frozen, AWS said. On one ml.g7e.2xlarge instance, the pretrained model left most answers empty; the adapted model answered every question with a supplied option.
What does this change for operations teams?
AWS said the run demonstrates adaptation to a new task and answer format, not a comparison with other models or an official ARFBench leaderboard result. AWS warns that Flamingo’s fixed-size representation bounds what reaches the language model.
It “does not give the complete system unlimited memory” and cannot guarantee that a brief event remains identifiable in a very long recording, AWS said. The 6 MB payload cap and the model’s useful analysis window remain separate constraints.
AWS advises testing representative normal and abnormal windows, retaining sampling resolution for short events, aligning related channels and including units, sampling intervals and observation periods in the context. The practical test is whether those reviewed, representative examples produce reliable explanations and recommended actions, AWS said.
For buyers, the next proof point is operational: those examples must validate the model before it enters the workflows it is meant to support. The launch removes integration friction, but it does not remove the operating burden of testing.
Source: unite.ai
More from Nathan Brooks
Show full bio
News editor covering regulation, policy and their impact on entrepreneurship, startups and venture funding at Business Bearings.
2,352 articles