ERNIE-Image on SGLang-Diffusion: Cache-DiT + ComfyUI Integration for 2.5x Inference Speedup

The Evolution of SGLang-Diffusion

In November 2025, LMSYS Org released SGLang-Diffusion — a landmark project that brought SGLang's high-performance inference engine to diffusion models. ERNIE-Image was among the first models to receive day-0 SGLang support.

By January 2026, after two months of intensive optimization, SGLang-Diffusion achieved 2.5x faster inference compared to its initial release. This was made possible by a stack of key technologies: Cache-DiT caching mechanism, Layerwise Offload, hybrid Sequence Parallel + Tensor Parallel, and deep ComfyUI integration.

For ERNIE-Image users, this translates directly to faster iteration cycles, lower latency in production, and the ability to run the 8B model on more modest hardware. In this article, we dive deep into the latest SGLang-Diffusion optimization stack that takes ERNIE-Image inference performance to new heights.

Cache-DiT: Diffusion Transformer Caching Acceleration

The Principle: Eliminating Redundant Computation

Diffusion model inference requires multiple denoising steps (50 for ERNIE-Image Base, 8 for Turbo). At each step, the model performs a complete forward pass on the entire image. However, research has shown that feature map changes between adjacent denoising steps are very small — meaning most computation is redundant.

Cache-DiT's core idea is elegantly simple: cache and reuse intermediate features from adjacent steps, skipping unnecessary computation. When feature changes are small enough, the cached result from the previous step is reused instead of recomputing. This "lazy evaluation" strategy can dramatically reduce inference time with negligible impact on final image quality.

Enabling Cache-DiT

Enabling Cache-DiT in SGLang requires just two environment variables:

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_SCM_PRESET=fast \
sglang generate --model-path=baidu/ERNIE-Image-Turbo \
  --prompt="A black-and-white Chinese rural dog running on grass" \
  --save-output

Parameter	Options	Description
SGLANG_CACHE_DIT_ENABLED	true / false	Enable/disable Cache-DiT
SGLANG_CACHE_DIT_SCM_PRESET	fast / balanced / quality	Cache strategy: speed-first, balanced, quality-first

Performance Improvements

Configuration	Inference Time	Speedup
No Cache-DiT	Baseline 100%	1.0x
Cache-DiT (quality)	~60%	1.67x
Cache-DiT (balanced)	~50%	2.0x
Cache-DiT (fast)	~37%	2.69x

📊 Based on LMSYS Org official benchmarks. Actual results vary by GPU model, image resolution, and step count.

Synergy with torch.compile

Cache-DiT is fully compatible with PyTorch's torch.compile. When used together, the acceleration effects stack:

import torch
import sglang as sgl
Enable torch.compile + Cache-DiT
sgl.cache_dit.enable()

model = sgl.ErnieImagePipeline.from_pretrained(

"baidu/ERNIE-Image-Turbo",

torch_dtype=torch.bfloat16

)

model.transformer = torch.compile(model.transformer, mode="reduce-overhead")
image = model("Night city skyline, blue neon lights, cyberpunk style").images[0]

image.save("cyberpunk_city.png")

torch.compile optimizes the computation graph through JIT compilation, while Cache-DiT reduces computation through runtime caching — they accelerate from different dimensions without conflict.

Hybrid Parallelism: Sequence Parallel + Tensor Parallel

For production deployment, single-GPU inference throughput may not meet high-concurrency demands. SGLang-Diffusion supports multiple parallelism strategies in combination:

Sequence Parallel

Splits the sequence dimension (patch/token sequence) of an image across multiple GPUs. Ideal for high-resolution image generation (ERNIE-Image supports up to 2048×2048).

Tensor Parallel

Splits model weights across multiple GPUs, with each GPU responsible for computing a portion of the parameters. Reduces per-GPU VRAM pressure.

Hybrid Configuration

SGLang supports any combination of Ulysses Parallel + Ring Parallel + Tensor Parallel:

sglang serve --model-path baidu/ERNIE-Image-Turbo \
  --tp 2 \
  --sp 2 \
  --host 0.0.0.0 --port 30000

This uses 4 GPUs: 2-way Tensor Parallel + 2-way Sequence Parallel. For ERNIE-Image-Turbo (8B parameters), a 4× RTX 4090 (24GB) configuration can achieve near-real-time batch inference.

Layerwise Offload: Low-VRAM Lifeline

For consumer GPUs with only 16GB or even 12GB VRAM, ERNIE-Image's 8B parameters (~16GB in bfloat16) are challenging to load entirely. SGLang-Diffusion's Layerwise Offload mechanism solves this:

from sglang.srt.managers.diffusion_offload import LayerwiseOffloadManager
from sglang.srt.models.ernie_image import ErnieImageDiT
offload_manager = LayerwiseOffloadManager(model, offload_per_layer=True)

model.transformer.enable_offload(offload_manager)

How it works:

1. Model weights remain in CPU memory (system RAM is plentiful)
2. During inference, layers are transferred one-by-one from CPU to GPU
3. After computation, results are immediately transferred back to CPU, freeing GPU VRAM
4. The process repeats for the next batch

Layerwise Offload enables ERNIE-Image to run on GPUs with 8GB VRAM (e.g., RTX 3070). While inference speed is reduced due to PCIe data transfer overhead, it makes local deployment accessible for low-VRAM users.

ComfyUI Integration: Using the SGLang Engine in ComfyUI

The SGLDiffusion ComfyUI Plugin

The SGLang team provides a ComfyUI custom node that replaces ComfyUI's built-in denoising forward pass with SGLang's optimized inference engine. This means you maintain full workflow flexibility in ComfyUI while benefiting from SGLang's performance improvements.

Installation:

1. Navigate to ComfyUI's custom_nodes/ directory
2. Clone the SGLDiffusion repository:

cd ComfyUI/custom_nodes/
git clone https://github.com/sgl-project/sglang.git
cp -r sglang/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion ./

3. Restart ComfyUI

Usage:

Replace ComfyUI's default KSampler node with the SGLDiffusion KSampler node. Configure:

• Model: Select the SGLang-hosted model endpoint
• Enable Cache-DiT: Check to enable caching acceleration
• Cache Mode: fast / balanced / quality
• LoRAs: Optional, load LoRA models from HuggingFace

Benefits with ERNIE-Image

Using the SGLang inference engine in ComfyUI provides 2-2.5x inference acceleration while maintaining full ERNIE-Image workflow compatibility:

• Rapid iteration: A 1024×1024 ERNIE-Image-Turbo image goes from ~1.5-2s to ~0.7-1s with Cache-DiT enabled
• Batch generation: Combined with Sequence Parallel, processing 4 images adds only 30-40% total time
• Advanced workflows: Complex pipelines with ControlNet + IP-Adapter + multi-pass refinement all benefit per-step

Production Deployment Best Practices

GPU Selection Guide

GPU Model	VRAM	Recommended Config	Expected Performance
RTX 4090	24GB	Single GPU, Batch=4	0.7s/img (Turbo, Cache-DiT)
RTX 5090	32GB	Single GPU, Batch=8	0.5s/img (Turbo, Cache-DiT)
2× RTX 4090	48GB	TP=2, High resolution	Real-time 2048×2048
4× A100 80G	320GB	TP=4, SP=2, Production	~2000 images/hour
RTX 3070	8GB	Layerwise Offload	Functional but slow

Deployment Commands

Basic deployment:

sglang serve --model-path baidu/ERNIE-Image-Turbo \
  --port 30000 \
  --host 0.0.0.0

Production with Cache-DiT:

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_SCM_PRESET=balanced \
sglang serve --model-path baidu/ERNIE-Image-Turbo \
  --tp 1 \
  --sp 1 \
  --port 30000 \
  --host 0.0.0.0 \
  --max-running-steps 64

Multi-GPU high concurrency:

SGLANG_CACHE_DIT_ENABLED=true \
sglang serve --model-path baidu/ERNIE-Image-Turbo \
  --tp 2 \
  --sp 2 \
  --port 30000 \
  --host 0.0.0.0 \
  --enable-mix-parallel

Performance Tuning Checklist

Optimization	Expected Gain	Difficulty
Enable Cache-DiT (fast)	2.69x	Low
Enable torch.compile	1.2-1.5x	Medium
Use SageAttention2/3	1.3-1.8x	Low
Enable Layerwise Offload (low VRAM)	Enables 8GB GPUs	Medium
Sequence Parallel (multi-GPU)	Linear scaling	High
Adjust batch size	Higher throughput	Low

Complete ERNIE-Image SGLang Capability Matrix

Feature	SGLang Status	Notes
ERNIE-Image Base (50 steps)	✅ Full	Day-0 SGLang support
ERNIE-Image Turbo (8 steps)	✅ Full	Distilled model optimized
Cache-DiT caching	✅ Full	Up to 2.69x speedup
torch.compile	✅ Full	JIT compilation optimization
LoRA loading	✅ Supported	Dynamic loading via API
Hybrid parallelism	✅ Supported	Ulysses + Ring + TP
SageAttention3	✅ Supported	Latest attention backend
Low-VRAM deployment	✅ Layerwise Offload	8GB VRAM usable
ComfyUI integration	✅ Custom node	Replaces native KSampler
FP8/INT4 quantization	✅ Basic support	Further VRAM reduction

SGLang-Diffusion vs Diffusers Deployment

Dimension	SGLang-Diffusion	Diffusers + PyTorch
Inference speed	2-3x faster (with Cache-DiT)	Baseline
Multi-GPU support	Native parallelism	Manual implementation needed
Dynamic LoRA loading	✅ API support	Requires restart
Production-grade serving	✅ Built-in	Requires additional framework
Learning curve	Medium	Low
Community ecosystem	Fast-growing	Mature but stable
ComfyUI integration	✅ Deep integration	Native support
Quantization support	FP8/INT4	FP8/INT4/GGUF

Summary

The pace of SGLang-Diffusion's evolution is remarkable. From its initial release in November 2025 to achieving 2.5x the inference performance of that initial release by January 2026, the SGLang team delivered a series of major updates — Cache-DiT integration, hybrid parallelism, LoRA support, ComfyUI plugin — in just two months.

For ERNIE-Image users, this means:

• Local users: Enable Cache-DiT for an instant 1.67x-2.69x speedup with zero additional hardware investment
• Low-VRAM users: Layerwise Offload makes the 8B model runnable on 8GB VRAM GPUs
• Production deployments: Hybrid parallelism + SageAttention + Cache-DiT can multiply service throughput several times over
• ComfyUI users: The SGLDiffusion custom node delivers high-performance inference within your familiar workflow

If you're still using the default Diffusers inference pipeline, now is the perfect time to migrate to SGLang-Diffusion — especially since Cache-DiT requires just two environment variables to enable, with zero model compatibility issues. It is, by far, the highest ROI optimization available for ERNIE-Image inference.