Kandinsky 6.0 Video

Sber's Kandinsky Lab released Kandinsky 6.0 Video, an open-weights foundation diffusion model family designed to generate temporally synchronized video and 44 kHz high-fidelity audio within a single unified pass. Standard open video generators produce mute video tracks that require separate Foley, ambient sound, or speech generation models in downstream post-production. Kandinsky 6.0 bypasses this pipeline bottleneck by coupling acoustic and visual generation within the diffusion process itself, enabling accurate environmental audio, sound effects, and lip-sync synchronization directly from text or image prompts.

The model is distributed under the permissive MIT license, granting unrestricted commercial and self-hosted deployment. The family ships across two main tiers: Kandinsky 6.0 Video Lite (3B parameters) and Kandinsky 6.0 Video Pro (29B parameters). Both models produce 5-second video clips at 24 frames per second (125 total frames) paired with uncompressed 44 kHz audio. Native generation handles Standard Definition (864x480) and High Definition (1280x720) outputs, which can be upscaled to 1080p Full HD using an accompanying plug-in super-resolution model.

Kandinsky 6.0 Video replaces conventional single-modality video transformers with a dual-stream CrossDiT (Cross-Diffusion Transformer) architecture. The backbone pairs a pretrained visual diffusion transformer stream with a dedicated audio diffusion transformer stream. These two pathways communicate continuously at every stage of the denoising trajectory through bidirectional cross-attention mechanisms. The cross-attention layers map temporal frame tokens to corresponding audio spectrogram patches, preserving semantic parity and millisecond-level alignment throughout the 125-frame diffusion process.

Training followed a multi-stage regimen designed to preserve unimodal generation fidelity while enforcing strict cross-modal cohesion:

• Audio Pretraining: The audio stream was initially initialized and trained from scratch on large-scale standalone audio and speech corpora to establish acoustic representations.
• Joint Audio-Visual Training: The audio stream was coupled to the visual backbone and jointly trained on paired video-audio datasets, enabling the cross-attention layers to align motion dynamics to audio events.
• Supervised Fine-Tuning (SFT) and Post-Training: Both streams underwent fine-tuning on high-aesthetic video and broadcast-grade audio, followed by reinforcement learning based alignment to eliminate temporal drift and acoustic artifacts.
• Distillation: Step-distilled versions (lite-distill and pro-distill) compress the standard 50-step diffusion process down to low-step regimes for low-latency generation.

To maintain compute efficiency across varying GPU generations, Kandinsky 6.0 integrates custom attention kernels:

• Hopper Architectures (H100, H200): Leverages FlashAttention 3 to maximize tensor core utilization across CUDA 13 environments.
• Ampere, Ada Lovelace, and Blackwell (RTX 30-series, 40-series, 50-series): Employs SageAttention 2.2 / SageAttention2++ for visual self-attention, reducing quadratic memory scaling during high-resolution latent decoding.

Kandinsky 6.0 Video targets production media generation workflows where audio-visual synchronization is mandatory:

• Text-to-Audio-Video (T2AV): Generates 5-second cinematic sequences from natural language prompts, producing realistic environmental Foley (footsteps, water splashes, engine rumbles) locked to the on-screen physical action.
• Image-to-Audio-Video (TI2AV / I2AV): Animates a static input image into motion while synthesizing a matching acoustic environment or dialogue track, utilizing the input image as a conditioned reference frame.
• Native Character Speech and Lip-Sync: In conversational prompts, the cross-modal attention maps phonemes to mouth shapes, producing synchronized lip movement without relying on external post-processing tools like Wav2Lip.
• Super-Resolution Upscaling: The modular kandinsky6-sr latent model scales 480p and 720p generations up to clean 1920x1080 Full HD footage without corrupting fine textural details or temporal coherence.

To run Kandinsky 6.0 Video locally, your hardware must handle both the dense visual transformer and the parallel audio generation stream. When running dense generative checkpoints, memory management during latent caching dictates whether a consumer workstation can process full sequences without out-of-memory (OOM) faults.

The non-distilled models run at 50 diffusion steps by default at 864x480 resolution (125 frames). VRAM consumption varies heavily based on whether you employ module offloading (enterprise cards) or sequential block offloading (consumer cards):

• Kandinsky 6.0 Video Lite (3B):
• BF16 Full Weights: ~8 GB VRAM dedicated to model parameters.
• Minimum VRAM (with block CPU offload): 12 GB VRAM (RTX 3060 12GB, RTX 4070).
• Recommended VRAM: 16 GB to 24 GB VRAM (RTX 4080, RTX 4090) for fast batching and higher native resolutions.
• Kandinsky 6.0 Video Pro (29B):
• BF16 Full Weights: ~58 GB VRAM just for weights.
• Minimum VRAM (with aggressive CPU offloading): 24 GB VRAM (RTX 3090, RTX 4090).
• Recommended VRAM: 32 GB to 96 GB VRAM (RTX 5090 32GB, RTX PRO 6000 Ada 48GB, or A100/H100 80GB).

Because full 29B FP16 weights exceed standard consumer capacities, choosing the best quantization for Kandinsky 6.0 Video is essential for sub-enterprise deployments:

• FP8 / INT8 Quantization: Loading the visual transformer in FP8 drops weight residency to ~29 GB, allowing the Pro model to execute on a single 32 GB GPU (such as an RTX 5090) with minimal offload overhead.
• Q4 / GGUF Formats: When adapting workflows for memory-constrained workstations, running 4-bit weights reduces the core footprint below 18 GB, allowing developers exploring how to run undisclosed model on consumer GPU setups to execute generation on 24 GB cards without thrashing system RAM.
• CPU Offload: Frameworks like Diffusers and vLLM-Omni use --enable-cpu-offload to swap transformer layers between system memory and GPU VRAM dynamically. This is required on any single 24 GB or 80 GB card running the full unquantized Pro pipeline.

Unlike text LLMs measured by tokens per second, Kandinsky 6.0 Video performance is benchmarked by total clip generation latency for a complete 5-second, 125-frame render (excluding disk writes):

For typical local practitioner workstations, the best GPU for Kandinsky 6.0 Video Pro is the RTX 5090 or RTX 4090. The RTX 5090's 32 GB buffer and memory bandwidth yield a complete 5-second SD clip in roughly 12.5 minutes on the full Pro model, while the distilled weights cut that generation time significantly.

Practitioners have three main pathways to run the model locally:

1. ComfyUI: The simplest interface for visual creators. Install kandinsky6 and kandinsky6-sr via ComfyUI Manager. Nodes handle cross-attention audio decoding and automatically route the output audio stream into the multiplexed MP4 container.
2. Official CLI Pipeline:

1   git clone https://github.com/kandinskylab/kandinsky-6.git
2   cd kandinsky-6
3   just setup
4   just download pro-distill
5   just generate "A sports car accelerating down an empty tunnel, engine roaring" --config kandinsky/configs/devices/rtx-4090.yaml --out clip.mp4

1. vLLM-Omni Serving: Kandinsky 6.0 is integrated into vllm-omni for headless multi-GPU inference and API hosting, utilizing HTTP multi-part requests to serve raw MP4 payloads directly to client applications.

While tools like Ollama dominate local text-only LLM deployments, video-diffusion architectures require ComfyUI, Diffusers, or vLLM-Omni backends due to their specialized multi-latent decoders and CUDA attention kernels.

Tencent's HunyuanVideo is currently one of the leading open video foundation models, featuring a massive dense transformer architecture. In purely visual tasks, HunyuanVideo delivers exceptional spatial consistency, prompt comprehension, and complex physics handling.

However, HunyuanVideo is strictly a silent video generator. Deploying it requires orchestrating a secondary audio generation model (such as MMAudio or AudioLDM2) alongside a tertiary lip-sync pipeline if characters speak. Kandinsky 6.0 Video Pro eliminates this multi-model synchronization pipeline entirely. While HunyuanVideo holds a slight edge in complex multi-subject visual scenes, Kandinsky 6.0 delivers higher production utility for creators who need turnkey audiovisual generation without manual sound design.

THUDM's CogVideoX-5B operates in a similar weight class to Kandinsky 6.0 Video Lite (3B). CogVideoX-5B uses an expert transformer with 3D VAE compression that runs comfortably inside consumer VRAM limits.

Kandinsky 6.0 Video Lite offers two concrete operational advantages over CogVideoX-5B:

• Audio Integration: CogVideoX generates mute clips, whereas Kandinsky Lite includes synchronized 44 kHz audio natively.
• Permissive Licensing: CogVideoX imposes custom commercial restrictions based on user scale, while Sber released Kandinsky 6.0 under the standard MIT license, allowing unrestricted commercial integration, local model fine-tuning, and offline distribution.