ERNIE-Image × ComfyUI Ecosystem Deep Integration: From Node Architecture to Custom Workflow Building
Three months have passed since Baidu open-sourced ERNIE-Image on April 15, 2026. The 8B-parameter Diffusion Transformer has not only dominated benchmark discussions but has also established a thriving ComfyUI ecosystem. Comfy-Org repackaged and released official workflow templates on launch day, YouTube creators contributed dozens of tutorial videos (the most popular reaching 17K views), and platforms like RunComfy, fal.ai, and Replicate added native support. The ecosystem has evolved from "will it run?" to "how to run it better."
Yet many users still struggle with fundamental questions: What are the four model components and how do they work together? What's the difference between Base and Turbo workflow templates? Which quantization format — GGUF or NVFP4 — fits your GPU best? How do you integrate ERNIE-Image into existing ControlNet or IP-Adapter pipelines? This guide builds a complete practical framework for ERNIE-Image in ComfyUI, starting from the node architecture.
Node Architecture
ERNIE-Image's ComfyUI node system consists of four core components, and understanding their collaboration is key to mastering the workflow.
Diffusion Model: ErnieImageTransformer2DModel, approximately 15GB. This is the core component responsible for actual image generation — it denoises random noise into target images through 50 steps (Base) or 8 steps (Turbo). Built on a single-stream DiT architecture, all attention computations happen within this module.
Text Encoder: Mistral3Model, approximately 7.2GB. Unlike most text-to-image models that use CLIP or T5 encoders, ERNIE-Image leverages Mistral3 — a 7B-parameter pure-text LLM. This gives it native support for long text understanding and complex instruction following, but also introduces higher VRAM overhead.
Prompt Enhancer (PE): Ministral3ForCausalLM, approximately 7.2GB. This is ERNIE-Image's most distinctive component — a dedicated 3B model that automatically expands short inputs into detailed Chinese descriptions. The PE is optional and can be disabled to save VRAM or retain full prompt control.
VAE: AutoencoderKLFlux2, only 161MB. ERNIE-Image reuses FLUX.2's VAE, inheriting its high-fidelity latent space. This choice provides solid decode quality but also means the model depends on FLUX's VAE characteristics.
The total combined size is approximately 29.5GB (BF16) at full precision, which can be significantly reduced through quantization.
Base vs Turbo
ERNIE-Image comes in two model versions, each with its own dedicated ComfyUI workflow template.
ERNIE-Image (Base): An SFT model requiring 50 inference steps. It excels in general-purpose capability and instruction fidelity — performing more reliably on complex scene composition, multi-object generation, and text rendering. Best for quality-first scenarios like commercial posters and comic book panels.
ERNIE-Image-Turbo: Optimized through DMD (Distribution Matching Distillation) and RL, requiring only 8 inference steps. It achieves measurably better speed and aesthetic quality, though with slightly reduced instruction precision compared to Base. Ideal for batch generation, rapid prototyping, and high-iteration workflows.
Switching between the two in ComfyUI is straightforward: load the corresponding safetensors file in the Load Diffusion Model node and load the matching workflow JSON. The node architecture is identical, with only sampling steps and recommended CFG Scale differing.
| Feature | Base | Turbo |
|---|---|---|
| Inference Steps | 50 | 8 |
| Relative Speed | 1x | ~6x |
| Instruction Following | Stronger | Good |
| Aesthetic Quality | High | Higher |
| VRAM Usage | ~18GB | ~18GB |
| Recommended CFG | 5.0 | 1.0 |
Template Workflow
ComfyUI's official template library includes two ERNIE-Image workflows: "Ernie Image Turbo: Text to Image" and "Ernie Image: Text to Image."
The standard node connection logic follows this pattern:
- Checkpoint Loader → Loads diffusion model + text encoder
- CLIP Text Encode → Input prompt (supports Chinese and English)
- Prompt Enhancer (optional) → Automatically expands prompts
- KSampler → Controls sampling parameters (steps, CFG, sampler type)
- VAE Decode → Decodes latent space to pixel images
- Preview Image / Save Image → Preview and save results
Key parameter settings:
- Base: steps=50, cfg=5.0, sampler=DPM++ 2M Karras
- Turbo: steps=8, cfg=1.0, sampler=DPM++ 2M Karras
- Resolution: Default 1024×1024, adjustable to 1024×768, 768×1024, etc.
- PE: Enabled by default, can be toggled in the Load PE Model node
The template workflow's greatest value is providing an immediately usable starting point — download models to the correct directories, load the workflow JSON, and start generating with just a prompt.
Getting Started
Setting up an ERNIE-Image ComfyUI workflow from scratch requires the following steps.
Step 1: Download model files. Get the four files from Comfy-Org/ERNIE-Image on HuggingFace and place them in ComfyUI's models directory:
ComfyUI/models/
├── diffusion_models/ernie-image-turbo.safetensors
├── text_encoders/ernie-image-te.safetensors
├── ernie-image/ernie-image-pe.safetensors
└── vae/ernie-image-vae.safetensors
Step 2: Update ComfyUI. ERNIE-Image requires ComfyUI v0.3.20+. Use the desktop client or pull the latest code from GitHub.
Step 3: Load the workflow. Select the ERNIE-Image template from ComfyUI's template library, or download the workflow JSON from the official docs and drag it in.
Step 4: Test generation. Verify the pipeline with a simple prompt:
"A black and white Chinese field dog, high-definition photography"
If PE is enabled, the model will automatically expand this into a detailed description.
Step 5: VRAM optimization. If you encounter OOM errors (sub-24GB GPUs):
- Disable Prompt Enhancer (don't load the PE node)
- Reduce resolution to 768×768
- Use enable_model_cpu_offload() or enable step-wise loading in ComfyUI
GGUF Quantized Workflows
The Unsloth team provides complete GGUF quantization for ERNIE-Image, compressing the model from 29.5GB to 8-16GB and enabling GPUs with 12-16GB VRAM to run smoothly.
The GGUF workflow differs from the standard workflow only in the model loading node: use the Unsloth GGUF Loader instead of the Checkpoint Loader. Sampler and VAE nodes remain unchanged.
Quantization level comparison:
| Level | Model Size | Min VRAM | Quality Loss |
|---|---|---|---|
| Q4_K_M | ~9GB | 12GB | Minimal |
| Q4_K_XL | ~11GB | 14GB | Nearly lossless |
| Q5_K_M | ~12GB | 16GB | Negligible |
| Q8_0 | ~16GB | 20GB | Lossless |
| F16 | ~29.5GB | 24GB+ | None |
Unsloth Dynamic 2.0 achieves 99.9% KL divergence retention at Q4_K_XL, meaning visually imperceptible quality loss.
Low-VRAM Strategies
For users with limited VRAM (12-16GB), several optimization options exist beyond GGUF quantization.
Option 1: PE offloading. The Prompt Enhancer consumes approximately 8GB of VRAM (BF16). Disabling it frees significant memory for the diffusion model — a zero-cost optimization for users who prefer full prompt control.
Option 2: Resolution downscaling. Dropping from 1024×1024 to 768×768 reduces VRAM usage by roughly 40%. Generated images can be upscaled afterward using ESRGAN or ComfyUI's Upscale Image node.
Option 3: CPU offloading. For sub-12GB GPUs, use enable_model_cpu_offload() to layer-load — keeping only one sub-model on GPU at a time. This reduces generation speed but ensures even 6GB cards can run the model.
Option 4: SGLang serving. For high-frequency generation workflows, consider deploying ERNIE-Image as an SGLang inference service and calling it from ComfyUI via HTTP API. This is ideal for multi-user shared environments.
Advanced Combinations
ERNIE-Image in ComfyUI is not limited to standalone text-to-image pipelines. It can be combined with other nodes for complex effects.
ControlNet Integration: While ERNIE-Image lacks official ControlNet support, you can experiment with ComfyUI's ControlNet nodes using weights from other models. Experimental support includes Canny edge detection and Depth maps — note these aren't officially trained adapters, so ControlNet weights should be lowered.
IP-Adapter Style Transfer: Load reference images through ComfyUI's IP-Adapter loader to transfer styles to ERNIE-Image outputs. A recommended approach is to generate the base image first, then use it as IP-Adapter reference input.
Image-to-Video Pipeline: Connect ERNIE-Image outputs to Wan 2.6 Image-to-Video nodes for a complete text → image → video pipeline within a single ComfyUI workflow.
Batch Processing: Use ComfyUI's Batch Prompt node or Queue Prompt feature to process prompt lists automatically. For e-commerce use cases, combine with ImageMagick or Python scripts for post-processing.
Community Resources
ERNIE-Image's ComfyUI ecosystem has developed a rich resource network over the past three months.
Templates and Workflows: The official template library provides Base and Turbo standard workflows. Community users have shared hundreds of custom workflows on OpenArt and Civitai, covering anime styles, comic panels, product rendering, and more.
Custom Nodes: Pixaroma has developed an ERNIE-Image-optimized custom node pack, including the Note node (rich text annotations) and Resolution node (quick aspect ratio and resolution switching). These significantly improve the workflow editing experience.
Prompt Resources: ERNIEImagePrompt.com is a community-maintained prompt collection site with hundreds of verified ERNIE-Image prompts organized by use case.
Cloud Platforms: RunComfy, fal.ai, and Replicate all provide one-click ERNIE-Image environments. RunComfy's AI Toolkit additionally supports cloud-based LoRA training.
Troubleshooting
Three months of community use have produced mature solutions for common issues.
Issue 1: Diagonal wave artifacts. Some users report 45-degree diagonal ripples at image edges, caused by Turbo's 8-step denoising handling high-frequency details insufficiently. Solution: increase steps to 12-15, or apply a lightweight denoising filter in post-processing.
Issue 2: Skin texture artifacts. ERNIE-Image can produce "reptilian" skin textures in portrait generation, stemming from overfitting to high-frequency textures in training data. Solution: add "soft skin" and "smooth texture" to prompts, or use CFG Scale 1.0 (Turbo mode).
Issue 3: Unstable LoRA training. Community feedback indicates ERNIE-Image LoRA training can be unstable for character consistency, especially facial features. Recommended baseline: rank 32, BF16, float8, AdamW8Bit, 3000 steps, 1024 preview resolution. High-quality captions for each image and enabled TE are crucial.
Issue 4: Model loading failures. "UNet model not found" or "Missing model keys" errors indicate an outdated ComfyUI version. Ensure ComfyUI ≥ v0.3.20 and model files are in the correct subdirectories.
Three months of real-world use have proven that ERNIE-Image on ComfyUI has moved well beyond the "does it run" phase. The real question now is "how to run it best." Mastering the node architecture, understanding quantization options, and leveraging community resources are the keys to unlocking this model's full potential.