Running a 27-billion-parameter model on a laptop has meant either renting cloud compute or accepting a model too small to trust with real work.

The cloud dependency for large local models just cracked open

Most teams that want capable on-device inference are forced to choose between a model large enough to be useful and hardware they actually own. That tradeoff is what Bonsai 27B is directly targeting.

A 27B model now fits in less space than most video files

PrismML Releases Bonsai 27B ships in two builds from PrismML: a ternary variant using weights of negative one, zero, or positive one at 1.71 bits per weight and a 3.9GB file size, and a 1-bit binary variant at 1.125 bits per weight sitting at 5.9GB. Both run multimodal inference with a 262,000-token context window, and both are released under Apache 2.0, meaning you pull the weights, load them locally, and run them without a usage agreement or API key.

The engineers shipping private models feel this first

• ML engineers deploying proprietary data pipelines who cannot route queries through a third-party API under any compliance framework.
• Researchers running long-context document analysis who pay per token on cloud providers and need a viable offline alternative.
• Mobile and edge developers who need a genuinely capable multimodal model that fits within device storage budgets under 6GB.

The vision tower is held at 4-bit precision separately, which preserves image understanding without inflating the core weight file.

Quantization at this scale has almost never held up this well

PrismML benchmarked across 15 tasks in thinking mode on H100 GPUs using EvalScope with vLLM, and the ternary build retained 94.6% of the full-precision model’s performance. For context, the standard “2-bit” GGUF build of the same model measures a true 2.8 bits per weight after accounting for scale overhead, meaning Bonsai 27B‘s 1.71-bit ternary variant is genuinely smaller while remaining competitive with what most teams currently call aggressive quantization.

What you can actually do with it today

• Run 262K-token document analysis entirely offline on a consumer laptop.
• Deploy multimodal inference on mobile hardware under a 6GB storage ceiling.
• Test reasoning tasks locally before committing to cloud inference costs.
• Integrate under Apache 2.0 into commercial products without licensing friction.

Both builds are available now on Hugging Face under PrismML’s account.

Pricing reflects the open-source bet

Free under Apache 2.0, weights hosted on Hugging Face.

The compression gap still shows on the hardest benchmarks

Retaining 94.6% of benchmark performance means 5.4% is gone, and on tasks where the full model scores near the top of its class, that gap is not always negligible.

The alternatives are heavier or older

BitNet b1.58 achieves comparable bit-widths but requires pretraining from scratch, making it inaccessible for teams that want to compress an existing capable model. Standard GGUF quantization at IQ2_XXS lands at 2.8 true bits per weight, a larger footprint with no multimodal support at that tier.

On-device AI is moving faster than most procurement cycles can track

The line between what runs in the cloud and what runs on a phone is shifting every quarter now, and the teams who know where that line sits will make better infrastructure decisions than those who don’t. We cover tools like this every Friday — subscribe here and we’ll send the best ones straight to you.