Optimize latent diffusion pipelines by offloading heavy VAE decoding to remote nodes, reducing VRAM footprint and improving throughput.
### Key Features
– **Decoupled Architecture**: Offload heavy Variational Autoencoder (VAE) decoding tasks from the primary generation GPU to a dedicated, lightweight remote endpoint.
– **VRAM Footprint Reduction**: Minimize peak memory usage during the final image generation phase, allowing for larger batch sizes and higher output resolutions on primary nodes.
– **Endpoint Pipelining**: Parallelize diffusion denoising steps of incoming requests while asynchronously processing VAE decoding for completed latents.
### Use Cases
– Scalable image and video generation services running Stable Diffusion models where VAE decoding acts as a GPU bottleneck.
– Distributed inference architectures looking to minimize cloud compute spend by aligning specific hardware profiles to denoising and decoding steps.
### Developer Pros & Cons
– **Pro:** Prevents Out-Of-Memory (OOM) crashes during high-resolution VAE passes by moving the workload off the primary GPU.
– **Con:** Introduces minor network latency and serialization overhead when transferring latent tensors between nodes.