Reinforced Self-Training (ReST)

ReST addresses a fundamental bottleneck in supervised fine-tuning: the scarcity and expense of high-quality labeled data. Rather than waiting for humans to annotate thousands of examples, the model itself generates candidate responses to a set of prompts. A reward model — or a human evaluation rubric — then scores each candidate, and only the top-scoring responses are retained for the next fine-tuning round. This "grow" and "improve" loop can be repeated multiple times, with each iteration raising the quality floor.

The enterprise appeal is significant. A company deploying a domain-specific assistant — say, a legal contract analyzer or an insurance underwriting advisor — can seed ReST with a few hundred expert-reviewed examples, let the model generate thousands of variants, score them against a domain rubric, and emerge with a finely calibrated model without engaging a large annotation team. The reward model itself can be a classifier trained on existing quality data, a stronger frontier model acting as a judge, or a structured rule-based scorer for deterministic tasks.

The operational consideration is the reward model's reliability. If the scorer has blind spots or can be gamed, the fine-tuning loop can amplify those failures through successive iterations — a phenomenon known as reward hacking. Enterprises should treat the reward model as a first-class artifact: version it, validate it on held-out data, and audit its scoring distribution before each ReST iteration to ensure alignment drift is caught early.