
NVIDIA's Nemotron-Labs-Sigma is a family of continuous diffusion language models; this 8B instruct variant generates text by denoising 16-D token embeddings block by block instead of predicting tokens left to right. It has an 8,192-token context, runs in bfloat16 on a single NVIDIA GPU at roughly 120 to 140 tokens per second on an H100, and ships as open checkpoints. The model targets reasoning and coding tasks and supports classifier-free guidance steering at inference.
A solid 8B-parameter dense language model from NVIDIA. A pragmatic middle-ground choice when you need open weights without a flagship-sized footprint. Newly released, so production-readiness is still being shaken out.
Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.
Access model weights, configuration files, and documentation.
No benchmark data available for this model yet.
Benchmark data sourced from: Hugging Face, Arena.ai, Artificial Analysis.
See how different quantization levels affect VRAM requirements and quality for this model.
| Format | VRAM Required | Quality | |
|---|---|---|---|
| Q2_K | 4.0 GB | Low | |
| Q4_K_MRecommended | 5.7 GB | Good | |
| Q5_K_M | 6.5 GB | Very Good | |
| Q6_K | 7.4 GB | Excellent | |
| Q8_0 | 9.4 GB | Near Perfect | |
| FP16 | 17.0 GB | Full |
The top devices for this model at 4-bit, ranked by fit and speed.
| Device | Grade | Speed | VRAM |
|---|---|---|---|
| AMD Radeon RX 7600 8GBAMD | SS | 40.9 tok/s | 5.7 GB |
| NVIDIA GeForce RTX 5060 Ti 8GBNVIDIA | SS | 63.7 tok/s | 5.7 GB |
| AMD Radeon RX 7700 XTAMD | SS | 61.4 tok/s | 5.7 GB |
| Intel Arc B580Intel | SS | 64.8 tok/s | 5.7 GB |
| NVIDIA GeForce RTX 4070NVIDIA | SS | 71.6 tok/s | 5.7 GB |
Energy cost on NVIDIA RTX A2000 (~41 tok/s, Q4_K_M) vs flagship API pricing.
| Source | Cost per 1M tokens |
|---|---|
Local (energy only)Nemotron-Labs-Sigma-8B-instruct on NVIDIA RTX A2000 · ~41 tok/s · 70W | $0.057 |
GPT-6.2 SolOpenAI · in $2.00 · out $10.00 | $4.40 |
Claude Fable 5.2Anthropic · in $8.00 · out $40.00 | $17.60 |
Gemini 4 NeonGoogle · in $3.00 · out $15.00 | $6.60 |
Grok 4.5xAI · in $2.00 · out $6.00 | $3.20 |
API prices blended at 70% input / 30% output.
Hardware amortisation not included. Run the full ROI calculator for payback math.
Cheapest current cloud rentals with at least 6 GB VRAM, refreshed hourly.
Advertising disclosure: we earn commissions when you shop through the links below.
| Option | Cost / GPU-hour |
|---|---|
NVIDIA Tesla V100 16GBVast.ai · Spot · 16 GB VRAM | $0.05 |
NVIDIA GeForce RTX 5060 TiVast.ai · Spot · 16 GB VRAM | $0.07 |
NVIDIA Tesla V100 16GBVast.ai · On-Demand · 16 GB VRAM | $0.07 |
NVIDIA GeForce RTX 4060 TiVast.ai · Spot · 8 GB VRAM | $0.08 |
NVIDIA GeForce RTX 3080Vast.ai · Spot · 10 GB VRAM | $0.08 |
Per-GPU rate across RunPod, the Vast.ai marketplace, DigitalOcean, and Vultr.
Spot tier is interruptible. Plan for restarts when comparing against on-demand prices.
Cumulative cost of buying the smallest device in our directory that runs this model at 4-bit, compared with paying flagship APIs for the same token volume.
Assumes 1 million tokens a day, a 70/30 input and output mix and $0.12 per kWh. The device only runs as long as the workload needs. Use the ROI calculator for your own numbers.
NVIDIA's Nemotron-Labs-Sigma-8B-instruct breaks away from standard autoregressive language generation. Rather than sampling tokens sequentially from left to right, this 8-billion-parameter dense model uses continuous diffusion over learned 16-dimensional token embeddings. It generates text in parallel blocks, iteratively denoising Gaussian noise into discrete tokens while conditioning on preceding context through a standard key-value (KV) cache. The result is an instruction-tuned model designed specifically for complex multi-step reasoning, mathematical problem solving, and code generation.
At the 8B scale, Nemotron-Labs-Sigma-8B-instruct operates in direct competition with established local weights like Meta's Llama 3.1 8B Instruct and Qwen2.5-7B-Instruct. However, its execution mechanics are fundamentally different. By operating in a continuous embedding space rather than a discrete token-probability space, the model provides fine-grained control over generation trajectories. Developers can adjust sampling parameters like classifier-free guidance (CFG) and score temperature at inference time, tuning the exact balance between structural determinism and creative diversity without re-prompting or retraining.
For engineers evaluating a local AI model with 8B parameters in 2026, Nemotron-Labs-Sigma-8B-instruct represents one of the first production-viable continuous diffusion alternatives to standard decoder-only transformers. It ships as open weights under the NVIDIA Nemotron Open Model License and runs in standard single-GPU environments.
The underlying backbone of Nemotron-Labs-Sigma-8B-instruct is a dense transformer utilizing grouped-query attention (GQA), SwiGLU activations, and rotary position embeddings (RoPE), configured with an 8,192-token context window. Unlike standard autoregressive models where the final layer projects hidden states directly to a discrete vocabulary distribution, Sigma couples its backbone to a continuous-diffusion generation head.
Key architectural mechanics include:
The 8,192-token context window accommodates full multi-turn developer prompts, technical documentation, and standard script files while maintaining manageable memory allocations for the KV cache during multi-step denoising.
Nemotron-Labs-Sigma-8B-instruct is supervised fine-tuned (SFT) for instruction-following, with deliberate optimization for tasks where parallel block coherence and embedding-space steering provide an algorithmic advantage.
Nemotron-Labs-Sigma-8B-instruct for coding workloads excels at structural infilling and block-level code generation. Because the model denoises groups of tokens simultaneously rather than committing strictly token-by-token, it demonstrates lower syntax error rates on structured programming languages like Python, C++, and Rust. It is particularly effective for generating boilerplate routines, complete functions, and API glue code where internal structural consistency across multiple lines is critical.
On multi-step reasoning benchmarks, standard autoregressive models often compound logical errors early in the sequence. Sigma addresses this through classifier-free guidance and distribution distillation. Developers can scale the sampling budget (pass@k) to explore diverse reasoning trajectories, significantly increasing the likelihood of identifying correct solutions on complex math and algorithmic logic tasks.
The model handles standard conversational interaction, structured JSON extraction, and tool calling within its 8,192 context window. When deployed as an execution agent, the ability to control output entropy via score temperature allows developers to force strict, schema-compliant outputs during data parsing tasks.
Because Nemotron-Labs-Sigma-8B-instruct uses a continuous diffusion head and custom sampling loops, the deployment pipeline differs slightly from typical text-generation models. The model runs via PyTorch (>= 2.4) and Hugging Face transformers using trust_remote_code=True. Standard samplers (DDPM/DDIM) are implemented in pure PyTorch, eliminating the need for specialized external C++ compilation or non-standard CUDA kernels.
The dense 8B parameter count dictates the base memory footprint. When sizing hardware, account for model weights, the 8,192 KV cache, and working memory for the parallel block buffers:
If you are planning how to run an 8B model on a consumer GPU, your setup will depend on the target precision:
The best quantization for Nemotron-Labs-Sigma-8B-instruct depends on whether you run standard PyTorch execution or an optimized runtime. Because continuous diffusion relies on precise numerical trajectories within the 16-D embedding space, quantizing the denoiser head too aggressively can degrade block convergence. For general consumer deployments, 8-bit weights (FP8/INT8) provide the best balance of speed and stability. If VRAM is constrained to 8 GB or 12 GB cards, 4-bit weight-only quantization allows execution, but users should test reasoning performance to verify that block denoising does not introduce token jitter.
In native bfloat16 on enterprise hardware like an NVIDIA H100, the model achieves roughly 120 to 140 tokens per second. On an RTX 4090, throughput depends heavily on the chosen Number of Function Evaluations (NFE) per block. When configured for few-step distilled generation (4 to 8 tokens per forward pass), the model achieves practical generation speeds comparable to standard autoregressive engines while reducing memory bandwidth bottlenecks during long sequence generation.
Evaluating Nemotron-Labs-Sigma-8B-instruct requires comparing it against mainstream 8B autoregressive options:
Choose Nemotron-Labs-Sigma-8B-instruct if you are a developer, security researcher, or ML practitioner seeking a steerable, local code-generation model that avoids traditional autoregressive failure modes and leverages continuous diffusion for structural tasks.

Explore the Provider
Aggregate stats, leaderboard, release timeline, and benchmark coverage across every NVIDIA model we track.

Explore the Family
The full Nemotron family leaderboard with sizes, benchmark scores, and a release timeline.

Check which of your devices can run this model and how fast.

Compare hosted per-token prices before you commit to local hardware.

Break-even math for self-hosting, renting a GPU, or paying per token.
Live status for the hosted APIs you might use instead of running this locally.