Shipping one speculative decoding drafter for all traffic types is like using the same compression algorithm for text and video — it works until the benchmark meets reality.

The averaged benchmark hides a real serving problem

Most speculative decoding projects optimize a single draft model against a blended benchmark, then deploy it everywhere. That drafter degrades on high-entropy conversation or stiff formal reasoning the moment real traffic diverges from that blend, which it always does.

Two drafters ship where one kept failing

Tencent Open is a torch-native, open-source training framework from Tencent that builds and trains two specialized draft models for Hunyuan 3 (Hy3) target models: an autoregressive multi-token prediction drafter for open-ended conversation, and a block-parallel DFlash drafter for code and mathematical reasoning. You pull the repo, select the drafter type matching your traffic profile, and train against your domain-specific data, producing a verifiable draft model ready to pair with the target for rejection-sampling inference.

Inference engineers are the first to close the gap

• ML engineers serving mixed LLM traffic who need domain-specific draft models without rebuilding training pipelines from scratch
• AI infrastructure teams running code generation endpoints who lose latency headroom whenever a general drafter mispredicts long identifier spans
• Research teams benchmarking speculative decoding who need clean, separated MTP and block-parallel baselines to compare against honestly

The core insight AngelSpec acts on is structural: high-entropy conversation and constrained formal reasoning pull acceptance rates in opposite directions as proposal depth increases, so one drafter cannot optimize both without sacrificing one.

Speculative decoding just split into a two-track discipline

Speculative decoding adoption accelerated sharply after Deepmind’s 2023 paper showed lossless 2-3x decode gains, and every major inference provider now ships some variant. AngelSpec signals that the next competitive front is not whether you use speculative decoding, but whether your drafter is actually matched to your traffic, and frameworks that ignore workload shape will lose latency benchmarks to those that do not.

What you can build with it today

• Train a conversation drafter tuned to your own chat traffic distribution
• Build a block-parallel drafter optimized for your code completion token patterns
• Benchmark MTP against DFlash on domain-specific datasets without custom scaffold code
• Integrate trained draft models with Hy3 target models for rejection-sampling inference

AngelSpec is open-source under Tencent’s release — pricing not listed, check our directory.

A torch-native framework this specialized will not stay niche

The honest tradeoff is that AngelSpec is built around Hy3 target models, so teams running other model families will need to adapt the training pipeline before it is directly useful.

For teams already on Hy3, Medusa offers multi-head drafting but does not separate training by workload type. SpecInfer handles tree-based verification but does not ship domain-split drafter training out of the box.

Drafter specialization is becoming the new inference optimization battleground

The era of one-size-fits-all speculative decoding is ending faster than most inference teams have planned for. We cover tools like this every Friday — subscribe here and we’ll send the best ones straight to you.