Overview
Vidu S2 is a real-time video model released by Shengshu Technology on September 15, 2026, just 69 days after Vidu S1. It comprises two models: Vidu S2-Avatar for continuous interactive digital character generation, and Vidu S2-Editing for real-time editing of input video streams. Both the web experience and the API opened to everyone on launch day, with no need to apply for early access.
S2-Avatar's improvements over the previous generation center on three areas: real-time output rises from 540P to 720P at 25 to 42 FPS; new reference images can be fed in at any point during video stream generation, letting the character pick up a specified object, change into a specified outfit, or enter a new scene; and motion capabilities expand from mostly talking to full-body, large-amplitude movements such as solo dancing and 2D and 3D animation. This kind of interaction also involves maintaining state after an action is completed—for example, "pick up the cup and then smile" requires the character to keep holding the cup while smiling. The model uses a VLM Agent to read user instructions and check output frames in temporal order, judging whether an action is complete, partially complete, or off-target, then adjusting subsequent prompts.
S2-Editing takes a continuous input video stream and performs four types of real-time editing with a single text instruction plus an optional reference image: style transfer, virtual try-on, character replacement, and background replacement. When a person raises a hand, turns, or the camera moves, the edited result must follow those changes. This relies on frame-aligned sparse attention, which lets a target frame read only the source video's frame at the same moment, preventing motion information from different moments from interfering with each other.
Beyond the two models, Shengshu Technology has extended its exploration into spatial video: converting real-time generated or real-time edited footage into synchronized left- and right-eye views with parallax, then streaming them to a VR headset for character interaction and image editing in spatial video. Currently this is mainly fixed-viewpoint binocular video, with panoramic spatial video still in the exploratory stage. The technical roadmap for the entire real-time pipeline is led by Zhang Jintao (a PhD student of Professor Zhu Jun), head of streaming video generation and inference at Shengshu Technology.
Key Features
- 720P real-time output at 25-42 FPS: Compared with Vidu S1, S2-Avatar improves from 540P to 720P with a real-time frame rate of 25 to 42 FPS, using a two-stage Backbone and Refiner architecture: the backbone handles motion and long-range temporal coherence at low resolution, while a single-step Refiner restores high-resolution detail.
- Reference images can be updated at any time during generation: A new reference image can be fed in at any point while the video stream is playing, letting the character pick up a cup beside them, change into a specified outfit, or enter a new scene. The interaction is never interrupted, and the model keeps track of the state of completed actions.
- Full-body, large-amplitude motion following: Training data now includes solo dance and 2D and 3D animation footage, combined with human preference optimization, enabling instruction following for large body movements such as dancing rather than being limited to talking-head performances.
- Four types of real-time video editing: S2-Editing supports real-time style transfer, virtual try-on, character replacement, and background replacement, keeping the edited result temporally aligned with the source video even as the person keeps moving and the camera moves.
- Frame-aligned sparse attention: A target frame interacts only with the source video's frame at the same moment, while the reference image stays visible to all target frames. The former provides motion and spatial structure, the latter guides appearance changes, reducing artifacts and tearing in real-time editing.
- SRF self-replay training: Self-Replay Forcing has the model first sample segments from its own generated trajectory, re-add noise, and perform causal replay training, learning the error relationships between adjacent segments to mitigate identity drift and motion breaks during long streaming generation.
- Spatial video for VR headsets: Converts real-time generated or real-time edited monocular footage into synchronized left- and right-eye views and streams them to a VR headset; S2-Editing supports both editing ordinary video first and then converting it to spatial video, and directly editing existing binocular video.
- S2-Avatar offline batch generation: There is also an offline interface for pre-planned content production, which can batch-generate digital human presenter videos with controllable action timing from images, scripts, or audio, using an asynchronous generation pipeline.
Use Cases
- Livestreaming and virtual selling: the character picks up a new product to show it at any point during real-time interaction and keeps holding it while continuing to talk
- Real-time virtual try-on: shoppers see how clothing fits their own movements before buying, lowering the cost of decision-making
- Real-time stylized capture: change the art style, clothing, subject or background on the fly from a camera feed, with no post-production rework
- Short-drama and ad asset proliferation: spin one original clip into multiple visual styles in real time, tailored to different audience segments
- Virtual production and mixed reality: feed the camera's perspective view into the editing pipeline and change the environment in real time
- VR spatial video experience: watch real-time interaction and real-time editing results inside a headset
Pros
- Fully open to all users at launch, with a web entry point, API platform and technical report
- S2-Avatar ranks first on all nine metrics reported in StreamAV-Bench, with 0.998 subject consistency and 0.993 background consistency
- S2-Editing scores 4.26 overall on the joint OpenVE and RefVIE evaluation, and 3.74 overall on Sparkle-Bench
- 720P real-time output at 25-42 FPS, covering large-amplitude motion such as dance
- Reference images can be updated at any point during generation, a clear expansion of interactive freedom over the previous generation
- All four editing tasks share the same interaction model of one text instruction plus one reference image, keeping the learning curve low
- Extends the real-time pipeline to spatial video for VR headsets, with a paper and a complete technical workflow
Pricing
The web experience is available at vidu.com/vidu-stream, and the API is accessible through the Vidu S2 documentation at platform.vidu.com. Pricing varies by model and usage mode: for real-time digital humans you can choose a full service bundle covering speech recognition, language model, speech synthesis and real-time communication, or use the component mode to plug into an existing voice system; offline generation runs through an asynchronous interface. For specific unit prices and new-user credits, refer to the official website and API platform announcements.
Summary
Vidu S2 moves video generation from generating a clip to controlling a continuously running video stream in real time, with each of the two lines targeting a clear use case: S2-Avatar for real-time digital human interaction and S2-Editing for real-time video editing. Compared with Vidu S1, this generation's changes go beyond resolution moving from 540P to 720P: the control target expands from a fixed digital human to dynamic reference images and full video streams, motion capability extends to full-body performance, and there is an added step of spatial video exploration for VR headsets. Teams working on livestreaming, virtual try-on, real-time virtual production or batch derivation of short-drama assets should start testing from Day 1; if the need is simply to generate a finished clip offline, a mature finished-video model like Vidu Q3 remains the easier choice.
Version History
- Vidu S2 launches with dual models S2-Avatar and S2-Editing, plus spatial video exploration (2026-09-15): Shengshu Technology released Vidu S2, just 69 days after Vidu S1. S2-Avatar raises real-time output from 540P to 720P at 25-42 FPS, supports updating reference images at any point during generation and improves tracking of large-amplitude motion such as dance; S2-Editing performs real-time style transfer, virtual try-on, character replacement and background replacement on an input video stream. On top of the two models, the team explored spatial video generation and editing for VR headsets. In evaluations, S2-Avatar ranked first on all nine metrics in StreamAV-Bench (0.998 subject consistency, 0.993 background consistency), while S2-Editing scored 4.26 on the joint OpenVE and RefVIE evaluation and 3.74 on Sparkle-Bench. The web experience and API access opened to all users on launch day.