Every time a researcher changes a reward estimator in a standard RL framework, they pay a tax in trainer glue, distributed backend wiring, and rollout scaffolding they didn’t ask to touch.

Algorithm iteration in agentic RL has been bleeding research time for years

Modifying a rollout scheme in frameworks like verl means navigating roughly 62,000 lines of RL code. Each experiment carries that overhead before a single gradient update runs.

Molt ships a codebase an AI coding assistant can read in full

NVIDIA AI Releases Molt, from NVIDIA’s NeMo team, composes Ray for async placement, vLLM for rollout, and NVIDIA AutoModel with FSDP2 for training into a single loop traced at approximately 8,600 lines of RL code. A researcher clones the repo, points it at a multi-turn tool-use or code-execution environment, and gets back on-policy training runs without forking any upstream dependency. Upstream improvements arrive as a container pin, not a rebase.

Frontier-adjacent labs feel this most immediately

• Post-training engineers at well-funded AI startups who rebuild rollout pipelines for every new reward model and need changes to stay local to one file.
• Enterprise AI research teams in finance, healthcare, or robotics who train agents against proprietary environments and cannot afford framework lock-in.
• Academic labs with multi-node H100 or H200 access running LLM-as-judge reward loops who need shipped Slurm scripts and a prebuilt container to start today.

The RL framework war is quietly being won on codebase size

OpenRLHF sits at 7,200 lines, slime at 25,000, and verl at 62,000 — and Molt is betting that the researcher who can read the entire framework in an afternoon will outiterate everyone else in post-training. As agentic tasks grow more complex and reward functions more bespoke, the framework that requires the least institutional knowledge to modify will absorb the most research workflows.

What you can run with it today

• Train multi-turn tool-use agents across 2 nodes of 8 H100 GPUs.
• Run the shipped geo3k recipe for vision-language RL environments.
• Wire in LLM-as-judge reward signals without touching the training loop.
• Distill on-policy behavior onto a smaller student model.

Apache 2.0, no licensing cost — hardware is the real gate, with recipes assuming 16 H100s minimum.

Molt is research infrastructure, not a production training service, so teams expecting managed scaling or multi-cloud abstractions will need to build those themselves.

verl is the most direct comparison at 62,000 lines with broader community adoption; OpenRLHF is closer in footprint but less opinionated about agentic multi-turn setups. Molt is the only one explicitly designed to be AI-coding-assistant readable from the entry point down.

The RL framework that fits in an AI assistant’s context window changes how research teams hire

We cover tools like this every Friday — subscribe here and we’ll send the best ones straight to you.