Step-Video-TI2V

Step-Video-TI2V is an open source, text-driven image-to-video (TI2V) foundation model developed by StepFun. Built directly upon StepFun's 30B-parameter Step-Video-T2V base, this model conditions video synthesis on both an input image (which serves as the initial frame) and a descriptive text prompt. It outputs a 102-frame sequence, roughly 5 seconds of footage at 540p resolution, with granular controls over physical motion amplitude and camera trajectories.

At 30 billion parameters, Step-Video-TI2V stands among the largest open weights image-to-video models available for local deployment. It sits in direct competition with heavyweights like HunyuanVideo-I2V and Wan2.1, targeting enterprise production pipelines, visual effects artists, and local AI enthusiasts who require deterministic motion control without being locked into proprietary SaaS platforms like Runway Gen-3, Kling, or Hailuo.

The release includes model weights, inference scripts, ComfyUI integration (ComfyUI-StepVideo), and native deployment support for both NVIDIA CUDA and Huawei Ascend architectures. For local developers, it represents one of the most capable video diffusion models released to date, though its massive dense architecture demands serious hardware planning.

Step-Video-TI2V utilizes a dense 30B-parameter Diffusion Transformer (DiT) architecture. Unlike Mixture-of-Experts (MoE) designs where only a subset of weights activate per token, dense architectures route the entire 30B parameter set through every denoising step. This translates to pure compute and memory bandwidth intensity: every parameter must be loaded into memory and processed across all attention heads for each inference step.

The model incorporates the first frame by encoding the source image into latent representations using StepFun's custom Video-VAE. These visual latents are concatenated directly along the channel dimension of the video latent sequence. Text embeddings, generated via an upstream multimodal text encoder, cross-attend into the DiT layers to guide thematic movement, style, and camera direction.

A critical engineering decision by StepFun is the decoupled inference strategy. To avoid running the text encoder, Video-VAE, and DiT on the same memory footprint simultaneously, the runtime isolates text feature extraction and VAE decoding behind lightweight local API servers (call_remote_server.py). The DiT itself can then leverage sequence parallelism (such as Ulysses parallelism across 4 or 8 GPUs) to split long sequence attention operations across multiple devices without hitting out-of-memory (OOM) ceilings during high-resolution, 102-frame generation.

Step-Video-TI2V is engineered for deliberate, production-ready motion control rather than random visual morphing. Its core capabilities center on three mechanical axes:

• Motion Dynamics Control (`motion_score`): Users can explicitly dial in the degree of movement. A low score (motion_score = 2) keeps the frame stable with subtle micro-movements, ideal for portraits, still life, or subtle environmental shifts. A moderate score (motion_score = 5, the default) yields natural human locomotion and fluid background dynamics. High values (motion_score = 10 to 20) trigger violent, fast-paced actions like martial arts, running, and explosive visual effects.
• Cinematographic Camera Control: The model interprets natural language prompts and directional tags to execute standard filmmaking camera moves. Supported movements include pan (left, right, up, down), tilt, zoom, dolly, rotation, tracking shots, orbit shots, and rack focus shifts.
• Aesthetic Retention and Anime Synthesis: The 30B parameter scale provides strong aesthetic fidelity. It handles stylized 2D anime, 3D CGI, and photorealistic cinematography without tearing lines or losing character consistency across the full 102 frames.
• Variable Aspect Ratios: While anchored around 540p resolution, the model supports standard cinematic landscape (544x992), square (768x768), and vertical portrait orientations without aspect distortion.

Common deployment use cases include programmatic asset generation for game development, automated B-roll synthesis in local video editing suites, anime keyframe interpolation, and privacy-critical commercial media generation where source imagery cannot leave on-premise infrastructure.

Deploying a local AI model with 30B parameters in a video diffusion pipeline is far more demanding than running an equivalent 30B language model. Step-Video-TI2V requires loading the 30B DiT alongside the decoupled text encoder and Video-VAE.

According to official benchmarks using 50 inference steps at native resolution (batch size = 1, without CFG distillation):

• Single-GPU Setup (768x768 or 544x992, 102 frames): Peak VRAM consumption reaches 75.5 GB to 76.4 GB. This requires a datacenter-grade 80GB GPU such as an NVIDIA A100 (80GB), H100 (80GB), or H200. On a single GPU, generating one 5-second clip takes roughly 929 to 1,061 seconds (15 to 17 minutes).
• Multi-GPU Parallel Setup (4x GPUs with Ulysses parallelism): Peak VRAM per card drops to 64.3 GB to 64.6 GB. Total inference latency drops dramatically to 251 to 288 seconds (around 4.5 minutes) for a 50-step generation.

If you are asking how to run 30B model on consumer GPU rigs, the straightforward answer for full-precision BF16/FP16 is that a standard single RTX 4090 (24GB) cannot run the baseline pipeline locally. Even dual RTX 4090 setups hit memory ceilings unless deep quantization and aggressive offloading are implemented.

For workstations and local production labs:

• Tier 1 (Enterprise / Production): 4x NVIDIA A100 (80GB) or 4x H100. This provides the memory headroom for parallel execution and delivers acceptable generation times.
• Tier 2 (Unified Memory Workstations): Apple Silicon machines such as the Mac Studio or MacBook Pro with an M3/M4 Max (128GB unified memory) or M2/M3 Ultra (up to 192GB). While inference speed will be constrained compared to dedicated Tensor Cores, unified memory bypasses PCIe bottlenecks and fits the full model weight allocation without crashing.
• Tier 3 (Ascend Hardware): StepFun provides native adaptation for Huawei Ascend platforms (e.g., Atlas 800 / 900 series via the Modelers community), offering an alternative to CUDA-locked infrastructure.

To run Step-Video-TI2V locally on consumer-accessible multi-GPU nodes (such as 2x or 3x RTX 4090s via ComfyUI), community quantization is essential.

• FP8 (Float8): Cuts model weight memory requirements nearly in half (~32GB to 36GB for the DiT alone). When paired with CPU offloading for the text encoder and VAE, an FP8 pipeline can fit on dual 24GB GPUs (48GB total pool) or a single workstation card like the RTX 6000 Ada (48GB). FP8 preserves virtually all motion fidelity and camera tracking accuracy.
• INT4 / Q4 Quantization: Community GGUF and AWQ/EXL2 formats (such as Q4_K_M) reduce the model footprint below 20GB. However, 4-bit quantization on video DiTs can introduce temporal artifacts, background warping, and reduced compliance with fine camera control tags. For video work, stick to FP8 whenever possible.

Unlike LLMs, which measure throughput in tokens per second, Step-Video-TI2V performance is measured in diffusion denoising speed (seconds per step) and overall frame synthesis latency. On 4x A100 GPUs, the model processes roughly 5 to 5.7 seconds per step, yielding an effective render speed of ~0.35 to 0.40 frames per second at 540p.

For developers seeking the fastest on-ramp:

1. ComfyUI (`ComfyUI-StepVideo`): This is the most practical interface for local artists and engineers. It simplifies graph management, handles node offloading, and allows chaining with LoRAs, ControlNets, and upscalers.
2. Native PyTorch / Torchrun: For automated backend services, run the decoupled API backend (call_remote_server.py) on a helper GPU, then execute run_parallel.py across your primary compute cluster using torchrun --nproc_per_node=4.

Evaluating Step-Video-TI2V requires comparing it against open-weight alternatives in the high-capacity video generation tier.

Tencent's HunyuanVideo represents one of the strongest open video architectures.

• Scale and Resource Efficiency: HunyuanVideo features a slightly smaller core DiT (around 13B parameters depending on configuration), making it significantly easier to run on consumer-grade hardware. An FP8 HunyuanVideo setup can fit within a single 24GB GPU using sequential offloading. Step-Video-TI2V, at 30B dense parameters, demands substantially more VRAM and compute power.
• Motion Quality: Where Step-Video-TI2V gains ground is motion amplitude fidelity. The native motion_score mechanism in Step-Video-TI2V provides predictable control over physical velocity that many 13B-class models struggle to maintain without frame degradation or subject melting.

Wan2.1 offers multiple model sizes (including 1.3B and 14B variants) focused on high accessibility.

• Hardware Accessibility: Wan2.1 is far more practical for single RTX 3090 / 4090 workstation operators. Its hardware barrier to entry is low, and inference runs significantly faster on consumer cards.
• Cinematography and Frame Consistency: Step-Video-TI2V is built for studio environments where multi-GPU infrastructure is available. Its 30B parameters excel at complex, multiperson dynamics, accurate camera rotations (orbit, rack focus), and detailed anime generation across a continuous 102 frames.

If you operate on a single consumer GPU, smaller alternatives like Wan2.1 or quantized HunyuanVideo are the pragmatic choice. If you possess enterprise multi-GPU hardware (or high-spec unified memory systems) and require fine-grained directorial camera control and high visual stability across 5 seconds of continuous generation, Step-Video-TI2V offers state-of-the-art open weights performance.