Innovation & Tech

AWS Adds Full Fine-Tuning to Serverless SageMaker, Hits 90% Correctness

AWS added full fine-tuning to serverless SageMaker on November 18, 2026, reporting 90.0% semantic correctness on 100K+ coding examples — while MMLU accuracy crashed to 3.6% at aggressive learning rates.

By Amara Osei

4 min read

AWS Brings Full-Weight Training To Serverless SageMaker Customization
AWS Brings Full-Weight Training To Serverless SageMaker CustomizationZOE-Animation-Studio / pixabay

What's News

  • AWS announced FFT support for SageMaker AI Serverless Model Customization on November 18, 2026.
  • AWS reported 90.0% semantic correctness in a Qwen3-4B coding test using 110,000-record Magicoder-Evol-Instruct dataset.
  • Learning rates of 5e-5 or higher caused catastrophic forgetting, with MMLU accuracy falling to 3.6%.
  • An LLM-as-Judge comparison favored LoRA at 81.5% over FFT's 79.5%.
  • AWS identified 2e-5 learning rate with 5% warmup and batch size 8 as the safest tested configuration.

Amazon Web Services announced on November 18, 2026 that SageMaker AI Serverless Model Customization now supports full fine-tuning (FFT), and reported 90.0% semantic correctness in a coding experiment built on more than 100,000 training examples. The same announcement carried a warning: aggressive hyperparameters triggered severe general-knowledge losses, with MMLU accuracy falling as low as 3.6%.

Customers supply a base model, a dataset and hyperparameters. The service allocates compute and manages checkpoints, with no GPU provisioning, cluster setup or distributed-training configuration. In AWS's SageMaker Python SDK example, switching methods means replacing TrainingType.FULL with TrainingType.LORA and leaving the rest of the job configuration unchanged.

What's the difference between FFT and LoRA?

FFT updates every weight in the model. LoRA freezes pretrained weights and inserts low-rank adapter matrices instead, cutting GPU capacity requirements but limiting updates to a smaller representational space. AWS argues that full-rank training can capture more complex domain adaptations as datasets grow — a function of training dynamics, not simply a larger count of trainable parameters.

Serverless operation changes who manages the infrastructure. It doesn't reduce the number of weights being trained, so the compute and quality economics still hinge on whether task-specific gains outweigh heavier training costs and any erosion of broader capabilities.

How did AWS test it?

AWS ran supervised fine-tuning (SFT) of a Qwen3-4B base model with identical prompts, targets and evaluation code across methods. The Magicoder-Evol-Instruct-110K dataset supplied coding instruction-response pairs covering algorithm design, debugging, data structures and systems programming. The split assigned 99,000 records to training, 5,500 to validation and 5,500 to a held-out test set.

The evaluation stack separated task performance from knowledge retention:

  • Amazon Nova Pro served as an LLM judge, reading generated code alongside the original prompt to assess logical correctness.
  • PRIMECODE measured similarity to reference solutions via deterministic BLEU, ROUGE-1/2/L and F1 metrics.
  • MMLU tested general knowledge across 57 academic subjects.

AWS reported the semantic-correctness score peaked at three epochs; longer training degraded every variant tested. Its radar comparison placed FFT ahead of LoRA and the base model on every displayed quality dimension, with the largest leads on BLEU and ROUGE-2 — evidence the gap wasn't confined to a single metric.

AWS said it ran tens of customization experiments across multiple base models, varying learning rate, batch size, warmup ratio, epoch count and dataset scale. Each run, from job submission to evaluation, took a few hours. The resulting models were automatically registered as versioned Model Packages in SageMaker Model Registry, ready for review or deployment.

Where does full fine-tuning fail?

AWS was explicit that FFT wasn't universally superior. Both FFT and LoRA underperformed the base model on MATH exact match. A separate LLM-as-Judge comparison favored LoRA at 81.5%, versus 79.5% for FFT.

Aggressive learning rates caused the sharpest damage. At 5e-5 or higher, MATH scores fell to 2–3% and MMLU accuracy dropped to 3.6% in AWS's tests — evidence of catastrophic forgetting, where domain-specific training overwrites pretrained knowledge. AWS's risk map identified 2e-5 as the safer setting for large datasets. A baseline combining that rate with 5% warmup and a global batch size of 8 best preserved general knowledge as measured by MMLU among the configurations tested.

Batch size also shaped retention. At a fixed learning rate, smaller batches produced noisier updates that behaved like a higher learning rate and raised forgetting risk. Larger batches cut the noise but also cut the number of optimization steps, potentially underfitting the model.

Which models can you fine-tune?

SFT with FFT is available for open-weight models including DeepSeek R1 Distill Qwen 1.5B, Google Gemma 4 26B A4B and NVIDIA Nemotron 3 Nano 30B (A3B), per AWS's supported-model list.

The FFT documentation flags the operational trade-offs. The method uses the same dataset formats as LoRA but requires more compute and longer training, produces larger artifacts containing full model weights, and is incompatible with continuous customization. On the upside, it removes adapter overhead at inference.

The launch makes deep model customization easier to run for teams without cluster expertise. Whether it pays off still depends on tuning discipline — AWS's own numbers show the line between a 90% correctness gain and a 3.6% MMLU collapse sits at the learning-rate setting.

Source: unite.ai

Share this article:

More from Amara Osei

Amara Osei

Show full bio

Senior reporter covering companies, deals and strategy in entrepreneurship, startups and venture funding at Business Bearings.

2,336 articles

Related articles

« Previous articleNext article »