Every millisecond your model spends waiting on a round-trip to a server is a millisecond your competitor’s local inference already answered.

Shipping AI features meant paying for inference servers

Web developers who wanted to run machine learning models in the browser were forced to either pay for server-side inference or accept the performance penalties of JavaScript-based runtimes. Both paths added cost, latency, or both.

The native runtime now lives inside the browser tab

Google Releases LiteRT.js takes Google’s existing LiteRT native runtime, compiles it to WebAssembly, and exposes it through a JavaScript API so developers load a .tflite model file and get GPU-accelerated inference running entirely on the user’s device. The input is any .tflite model; the output is on-device predictions served through CPU via XNNPACK, GPU via WebGPU, or experimental NPU via the WebNN API. Inference never leaves the browser, which means no data leaves the user either.

ML engineers building for the web feel this first

• Front-end engineers shipping computer vision features who are tired of paying per-inference API costs and managing backend scaling.
• Privacy-focused product teams at healthcare or fintech companies where sending raw input data to a server creates compliance problems.
• Mobile web developers who already deploy .tflite models on Android or iOS and want a single model file that works on web without retraining or converting formats.

The all-or-nothing delegation rule is worth knowing: if a model cannot be fully run on the selected accelerator, LiteRT.js falls back to WebAssembly CPU execution rather than splitting the graph, which keeps behavior predictable but means partial GPU offloading is not an option.

TensorFlow.js just lost its main argument for web ML

Google’s own benchmarks show LiteRT.js running up to 3x faster than existing web runtimes, including TensorFlow.js, across classical computer vision and audio processing workloads. As WebGPU support widens across browsers, the performance gap between native apps and web apps running local inference will close faster than most teams have planned for.

What engineers can do with it today

• Run .tflite image classification models in-browser with no server round-trip.
• Deploy the same model file used on Android directly to a web app.
• Switch inference backends between CPU, GPU, and NPU with a single config change.
• Build offline-capable AI features that work without an internet connection after first load.

LiteRT.js is open source and available now through Google’s GitHub; pricing is not applicable for the library itself, though WebGPU availability depends on the end user’s browser and hardware.

One real constraint before you commit

Operator coverage on GPU and NPU is narrower than on CPU, so complex custom models may silently fall back to slower WebAssembly execution without obvious warning.

The alternatives worth knowing

TensorFlow.js remains the most documented web ML library and has broader community tutorials, but its JavaScript kernels are the exact performance ceiling LiteRT.js was built to break through. ONNX Runtime Web offers a similar WebAssembly and WebGPU approach for .onnx models, making it the closest structural competitor for teams not already on the TFLite ecosystem.

The server-side inference default is losing ground fast

Tools like this are redrawing where the compute actually happens, and the implications for cost, privacy, and product architecture are significant. We cover tools like this every Friday — subscribe here and we’ll send the best ones straight to you.