VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

Qualitative showcase of VINCIE-NExT across diverse video editing scenarios. Our method enables temporally consistent stylization, dynamic background replacement, identity and attribute editing, scene transformation, and object removal, while preserving motion, structure, and visual coherence across frames.

Rather than modeling the edited video directly from the source video and instruction, which demands large-scale paired video data, VINCIE-NExT routes editing intent through the image domain, where mature large-scale editing priors are readily available:

Video → Image → Image → Video

V→I extracts a representative keyframe from the source video, I→I applies the instruction-guided edit at the image level, and I→V recomposes the edited image into a temporally consistent target video. The whole chain is flattened into one interleaved token sequence and modeled by a single Diffusion Transformer, so the image pair (I^s, I^t) acts as an in-context visual demonstration, a pixel-grounded appearance blueprint for every output frame. The backbone is a 3B MM-DiT initialized from the text-to-video model of VINCIE, shared by all sub-tasks and both inference stages.

Overview of VINCIE-NExT. Video editing is decomposed into a structured chain of sub-tasks, and the image editing pair serves as an in-context visual demonstration that transfers image editing priors to video generation via a unified diffusion backbone.

Joint Training on Multi-Source Data

Because each sub-task is a valid suffix of the interleaved sequence, every sub-task can draw from independently collected data: abundant image editing pairs for I→I, image-conditioned video generation data for I→V, and full-chain samples, all supervised by a single diffusion objective with one set of parameters. We further build video-based in-context image editing (V↔I↔I) data: GPT-4o annotates editing instructions for raw videos, which Seedream 4.5 applies to keyframes, yielding (source video, frame, edited frame) triplets without any paired video editing supervision. Because the source video stays attached as context, each sample teaches exactly the V→I→I sub-step whose output is reused as the demonstration for V→I→I→V, so image editing ability transfers into video editing rather than remaining a standalone skill.

In total, training uses 1.20M image editing pairs (I2I, from OmniEdit), 2.45M paired video edits (V2V*, from OpenVE), and 1.63M V↔I↔I triplets.

TDF3D-RoPE

In-context editing poses two conflicting positional demands: the model must distinguish tokens from different shots (e.g., I^s vs. I^t) while recognizing spatially corresponding patches across shots to propagate edits pixel-faithfully. Time Dual-Frequency 3D RoPE resolves this by fusing two 3D frequency tensors via element-wise addition: an inter-shot component with monotonically increasing global offsets that makes every shot distinguishable, and an intra-video component with zero global offset, so the same spatial location in any two shots shares identical (h, w) positions, enabling content-driven attention and pixel-faithful edit propagation. We also apply Temporal Position Randomization (TPR) to image training data, sampling the temporal index uniformly instead of fixing it to zero, so that the model grounds correspondences in spatial content rather than temporal identity.

Chain-of-Editing (CoE)

At inference, the decomposition is instantiated as two sequential diffusion stages. Stage 1 (V→I→I) generates the edited keyframe from the source video and its temporal mid-frame, processing visual segments frame-independently to match the spatial nature of image editing. Stage 2 (V→I→I→V) generates the edited video conditioned on the source video, source keyframe, and the Stage 1 latent, which is inserted directly in latent space without a VAE decode/re-encode cycle; the same backbone now attends jointly across frames to model temporal dependencies. Because each stage is an independent diffusion process over an explicit interleaved context, the chain provides principled test-time scaling: additional compute (or a stronger external image editor) can be spent to improve editing fidelity without retraining.

Evaluation on OpenVE-Bench (431 clips, eight editing categories), scored by Gemini 2.5 Pro on a 1–5 scale. VINCIE-NExT achieves an overall score of 3.08, surpassing all open-source methods (+0.59 over OpenVE-Edit, ~24% relative gain; +48.8% relative over the in-context video editor ICVE) and closing the gap to the proprietary Runway Aleph to 0.57, while substantially reducing the dependence on large-scale paired video editing data. Global Style is the standout category (4.17, above Runway Aleph's 3.72), and Local Remove improves from 1.85 to 3.24 over OpenVE-Edit as edits are routed through a mature image editor. Camera Edit remains the main limitation, since novel-viewpoint synthesis is a geometric operation outside the appearance-editing chain.

Training on V↔I↔I data alone already enables meaningful video editing, and CoE provides consistent gains when chain-structured data is present. The full combination of all three data sources with CoE achieves the best overall score.

A Pure V2V model (same backbone and V2V data, no decomposition or CoE) scores only 2.32, below OpenVE-Edit (2.49), which trains on a superset of our V2V data, so the gains do not come from a stronger backbone. Full data + CoE reaches 2.65 (+14.2%), and +74.1% on Creative Edit, a category with no V2V supervision, while adding only ~6% visual training tokens.

A blinded pairwise human study on all 431 clips with 10 raters (Fleiss' κ = 0.721) also prefers full data + CoE over V2V*-only + CoE on every dimension, most clearly Prompt Following (38% vs. 24%) and Consistency (28% vs. 9%).

Thanks to the modular I→I interface, replacing self-editing in Stage 1 with a stronger external image editor (Qwen-Image-Edit or Nano Banana) yields consistent improvement: advances in image editing transfer directly to video without retraining.

1. Global Stylization and Compositional Editing

Global scene stylization (left, summer-to-snow) and localized compositional editing (right, injecting fire/water/wind), both maintaining temporal consistency without task-specific fine-tuning.

2. Chain-of-Editing vs. Direct Inference

Comparison of V2V direct inference, CoE with self-editing, and CoE with an external image editor on subject replacement (left) and removal (right). Direct inference produces flickering and ghosting; CoE eliminates this instability by anchoring to a reference image, and routing through an external editor further improves semantic accuracy with no retraining.