Overview
JoyAI-Video-Edit is a real-time streaming video editing system open-sourced by JD.com, officially released in August 2026. Based on an autoregressive diffusion architecture, it processes video at 30 frames per second in 720P resolution, supporting causal frame-by-frame editing of live streams or uploaded videos, outputting results in real time without waiting for the full video. Its core capability lies in executing various editing operations through natural language instructions, such as subject replacement, local modification, background change, and style transfer. The official release includes deployment code, a technical report (arXiv:2608.03974), an online demo, and model weights, all under the Apache 2.0 license, providing a new open-source solution for the video editing field.
Key Features
- Real-time streaming processing: Uses a causal video VAE that encodes only current and past frames, enabling frame-by-frame real-time editing of live streams or uploaded videos, with end-to-end processing speed of 30.19 FPS at 720x1280 resolution (tested on a single NVIDIA B200).
- Natural language instruction editing: Users can describe editing needs in natural language, and the system automatically performs subject replacement (clothing change, hairstyle change, character transformation), local editing (removing specified objects), background change, style transfer (watercolor, oil painting, cyberpunk), and motion modification.
- Reference image-guided editing: Supports uploading reference images to guide the editing process, making results more aligned with user-specified visual styles or content characteristics.
- Efficient autoregressive diffusion architecture: Employs an MLLM condition encoder and a 16B-parameter multimodal diffusion transformer (MMDiT), combined with ARMD distillation to compress denoising steps to two, and introduces long-horizon autoregressive distillation and bounded KV state inference to improve inference efficiency.
- Complete open-source resources: Provides deployment code, technical report, online demo (joyai-labs.jd.com/v2v/), and model weights on Hugging Face (jdopensource), all under the Apache 2.0 license, facilitating developer use and secondary development.
Use Cases
- Real-time clothing or background change in live streaming to enhance interactivity
- Quick removal of specified objects in video post-production without manual frame-by-frame processing
- Converting ordinary videos into artistic styles such as watercolor, oil painting, or cyberpunk
- Hairstyle or character transformation for creative content generation
- Reference image-guided video editing for consistent batch processing
Pros
- Outstanding real-time processing capability, suitable for low-latency scenarios like live streaming
- Natural language instructions simplify operation, lowering the barrier to video editing
- Supports multiple editing types with broad functionality coverage
- Open-source license is friendly, with complete code and weights publicly available
- Innovative technical approach, integrating autoregressive diffusion and causal VAE
- Online demo allows quick experience, with high community attention (about 1600 stars on GitHub)
Pricing
The official pricing information has not been announced; model weights and code are free and open-source. Usage costs are subject to the official website and GitHub repository instructions.
Summary
JoyAI-Video-Edit is a real-time streaming video editing system open-sourced by JD.com, using an autoregressive diffusion architecture to achieve causal frame-by-frame processing at 30 FPS in 720P, supporting multiple editing operations driven by natural language. Its open-source resources are complete, and the technical approach is efficient, suitable for live streaming and creative video processing scenarios. The main current limitation is the high VRAM requirement of the 16B model; support for consumer-grade GPUs is still in progress, and the full training pipeline has not been released.