Prompt

结合这个图，帮我画一个qwen3.5的前向结构图flowchart TB %% ============ ① Embedding & Multimodal Fusion ============ TXT(["文本 token ids<br/>(B, L_t)"]) IMG(["图像 patches<br/>(N_p, 3, 16, 16)"]) VID(["视频 patches temporal=2<br/>(N_v, 3, 16, 16)"]) subgraph EMB["① Embedding & Multimodal Fusion"] direction TB TE["text Embedding lookup<br/>248320 × 4096 (BF16)"] VE["Vision ViT 27L + 2×2 Merger<br/>1152 → 4096 (见 §8, BF16)"] FUSE["在 <|image_pad|> / <|video_pad|><br/>占位符位置 scatter 视觉 token"] TE --> FUSE VE --> FUSE end TXT --> TE IMG --> VE VID --> VE FUSE --> POS["② M-RoPE position ids<br/>section = [11, 11, 10] (T, H, W)<br/>仅生成 ids,实际旋转在 §6 内做"] POS --> X["x ∈ (B, L, 4096)"] %% ============ ③ 60-Layer Hybrid Backbone ============ X --> STK["③ 60 × Hybrid Decoder Layer (见 §3–§7)<br/>[ GDN × 3 → GAttn × 1 ] × 15<br/>每层 = RMSNorm → Mixer → ⊕ → RMSNorm → MoE → ⊕"] STK --> FN["④ Final RMSNorm"] FN --> H["h ∈ (B, L, 4096)"] %% ============ ⑤ Output heads ============ H --> LMH["⑤ LM Head 4096 → 248320 (tied = false)"] H -. speculative .-> MTP["⑤′ MTP Block ×1 → 复用 LM Head (见 §9)"] LMH --> OUT(["next-token logits (B, L, 248320)"]) MTP -. -> OUT

Result

GPT Image-2 - Text to Image

Settings