实时视频版「Nano Banana」来了！160亿参数重磅开源
Ranking
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - JD.com open-sourced JoyAI-Video-Edit, a 16B-parameter streaming video editing model that edits live video at 720P/30 FPS with 226 ms latency, claiming to be the first to combine streaming architecture, real-time speed, and usable quality. It shifts video editing from offline batch rendering to interactive, "edit-while-playing" workflows.
- Architecture: MLLM conditioning encoder + causal video VAE + 16B multimodal diffusion Transformer, trained/deployed as an autoregressive diffusion editor; SA-DMD distillation cuts denoising from ~10+ steps to 2. On one Nvidia B200: 22 ms VAE encode, 185 ms DiT denoise, 19 ms decode → 226 ms request-to-response, 30.1 FPS end-to-end.
- Unbounded duration via "bounded KV state inference": only recent chunks plus the first frame are retained, giving fixed compute/memory regardless of stream length, with training tuned for stability under limited memory to curb drift.
- Benchmarks: OpenVE-Bench total 3.60 vs streaming baselines SANA-Streaming 2.62, LiveEdit 2.00, XMax-X2.0 1.87, StreamDiffusionV2 1.23 — and within range of offline commercial Runway Aleph (3.45) and PixVerse V6 (3.05). On the team's own LongV2VBench it ranks first in all five categories (3.30, +1.59 over XMax-X2.0) at 30.19 FPS; human blind preference 81–90% vs streaming rivals, but only 48% vs 44% against offline Bernini-R.
- Positioning: prior streaming editors stayed small (1.3B–2B, 480P) for speed; JD targets e-commerce livestream use cases (reference-image-guided RV2V virtual try-on) and embodied-AI data generation — replacing multi-model human-hand-removal/inpainting/robot-arm-rendering pipelines with a single real-time pass, alongside its JoyAI-VL-Interaction / Talker / RA model matrix. Code, weights, report, and demo released on GitHub and Hugging Face (Aug 5).
Sources (1)
实时视频版「Nano Banana」来了！160亿参数重磅开源
TL;DR - JD.com open-sourced JoyAI-Video-Edit, a 16B-parameter streaming video editing model that edits live video at 720P/30 FPS with 226 ms latency, claiming to be the first to combine streaming architecture, real-time speed, and usable quality. It shifts video editing from offline batch rendering to interactive, "edit-while-playing" workflows.
- Architecture: MLLM conditioning encoder + causal video VAE + 16B multimodal diffusion Transformer, trained/deployed as an autoregressive diffusion editor; SA-DMD distillation cuts denoising from ~10+ steps to 2. On one Nvidia B200: 22 ms VAE encode, 185 ms DiT denoise, 19 ms decode → 226 ms request-to-response, 30.1 FPS end-to-end.
- Unbounded duration via "bounded KV state inference": only recent chunks plus the first frame are retained, giving fixed compute/memory regardless of stream length, with training tuned for stability under limited memory to curb drift.
- Benchmarks: OpenVE-Bench total 3.60 vs streaming baselines SANA-Streaming 2.62, LiveEdit 2.00, XMax-X2.0 1.87, StreamDiffusionV2 1.23 — and within range of offline commercial Runway Aleph (3.45) and PixVerse V6 (3.05). On the team's own LongV2VBench it ranks first in all five categories (3.30, +1.59 over XMax-X2.0) at 30.19 FPS; human blind preference 81–90% vs streaming rivals, but only 48% vs 44% against offline Bernini-R.
- Positioning: prior streaming editors stayed small (1.3B–2B, 480P) for speed; JD targets e-commerce livestream use cases (reference-image-guided RV2V virtual try-on) and embodied-AI data generation — replacing multi-model human-hand-removal/inpainting/robot-arm-rendering pipelines with a single real-time pass, alongside its JoyAI-VL-Interaction / Talker / RA model matrix. Code, weights, report, and demo released on GitHub and Hugging Face (Aug 5).