NVIDIA-Nemotron-Labs-Dory1.0-4B

NVIDIA-Nemotron-Labs-Dory1.0-4B is a dense, 4-billion-parameter open-weights language model developed by NVIDIA and released on October 16, 2026. Designed specifically to challenge the traditional tradeoff between parameter count and reasoning depth, Dory1.0-4B introduces a looped latent-recurrent normalized-GPT architecture. Instead of scaling up raw parameter volume to tackle complex coding and logic, it loops input representations through a shared recurrent block at runtime. This design allows a compact 4B checkpoint to match the problem-solving depth of much larger models while maintaining a minimal static memory footprint.

For local practitioners, Dory1.0-4B establishes a compelling standard for what a local AI model with 4B parameters in 2026 can achieve. Traditional small models struggle with multi-step software engineering tasks and long-horizon logic. Dory circumvents this limitation by allocating additional computational passes per token in latent space rather than generating bloated chain-of-thought token streams. Licensed under the permissive OpenMDW-1.1 license for both commercial and non-commercial use, it ships with native weights in BF16, MXFP8, NVFP4, and Q5_K_M formats.

The model is English-only and text-only, targeted squarely at software engineering, complex instruction-following, structured function-calling, and deep reasoning. With a massive native context window of 262,144 tokens (256K), developers can deploy Dory1.0-4B directly into codebases, documentation hubs, and autonomous agent loops that previously demanded 14B or 32B models.

Dory1.0-4B breaks away from standard unidirectional transformer stacks. Its architecture is built on a normalized-GPT (nGPT) backbone organized into three distinct layer groups: an 8-layer Input block (I), a 20-layer Recurrent block (R), and an 8-layer Output block (O).

The primary architectural mechanism is latent recurrence:

• The Recurrent Loop: When a token passes through the 8-layer input group, it enters the 20-layer recurrent block. Rather than passing through once, the runtime execution engine cycles the hidden states through this recurrent block multiple times (typically 1, 2, or 4 loops) before routing the state through the final 8-layer output block. The initial input is reinjected into every loop cycle to maintain contextual grounding.
• Elastic Inference Compute: Because the recurrence happens within the same shared 20 layers, the model checkpoint size remains strictly 4B parameters. You can scale inference compute dynamically per request: allocate 1 loop for rapid chat responses, 2 loops for standard code generation, or 4 loops for complex reasoning. No retraining, fine-tuning, or model swapping is required.
• Hybrid Attention: The model interleaves sliding-window attention (SWA) with full-attention layers. This design mitigates the quadratic compute penalty across its 262,144-token context window while retaining precise global token recall.
• Last-Loop KV Caching: Standard recurrent models multiply KV cache storage by the number of recurrent iterations, which rapidly exhausts GPU memory. Dory solves this with last-loop KV caching, preserving only a single cache instance per recurrent attention layer. As a result, running 4 loops consumes no more KV cache VRAM than running a single loop.
• Native Low-Precision Pretraining: Rather than applying post-training quantization, NVIDIA pretrained the normalized-GPT backbone natively in NVFP4 (NVIDIA 4-bit floating point) before running continuation in BF16, preserving numerical stability at low bit-widths.

Dory1.0-4B is built for technical workloads where compact models historically fail. Its reasoning and agentic profile make it an exceptional engine for local developer workflows.

Using NVIDIA-Nemotron-Labs-Dory1.0-4B for coding provides performance that punches far above its weight class. On the vendor-reported SWE-bench Verified benchmark, Dory1.0-4B demonstrates the raw power of latent recurrence:

• 1 Loop: 50.3%
• 2 Loops: 59.5%
• 4 Loops: 61.8%

At 4 loops, Dory1.0-4B approaches the coding accuracy of models more than twice its physical size, such as Qwen3.5-9B. It can parse multi-file repository contexts, diagnose tracebacks, generate functional patches, and adhere to strict language syntax across full code repositories loaded into its 256K context.

The model is tuned for deterministic function-calling and structured JSON generation. It works reliably in agentic loops where tools must be invoked consecutively. Its lower parameter count delivers faster time-to-first-token (TTFT) during prompt ingestion, which keeps local agent orchestration responsive.

Because the model handles up to 262,144 tokens, it can ingest full API specifications, enterprise policy manuals, or architectural blueprints in a single prompt. Latent looping allows it to synthesize fragmented facts scattered across large context spans without hallucinatory drift.

To run NVIDIA-Nemotron-Labs-Dory1.0-4B locally, you must account for two computational variables: static parameter memory and dynamic loop execution time.

Thanks to last-loop KV caching, your VRAM budget depends primarily on model quantization and context length, not on the loop count:

• NVFP4 (Native 4-bit): ~2.5 GB weights. Fits comfortably inside 6 GB to 8 GB VRAM with 32K context. Requires an NVIDIA Ada Lovelace (RTX 40-series) or Blackwell architecture GPU for hardware acceleration.
• Q4_K_M / Q5_K_M GGUF: ~2.8 GB to 3.4 GB weights. The best quantization for NVIDIA-Nemotron-Labs-Dory1.0-4B on consumer hardware lacking NVFP4 support. Fits inside 8 GB VRAM with ample room for 64K context.
• BF16 (Unquantized): ~8.5 GB weights. Requires a minimum of 12 GB VRAM for short contexts, or 16 GB to 24 GB when extending past 64K tokens.

For full 262,144-token context allocation, plan for an additional 6 GB to 10 GB of VRAM strictly dedicated to the KV cache, depending on cache precision settings (FP16 versus FP8/Q4 cache).

Wondering how to run 4B model on consumer GPU setups? Dory1.0-4B is exceptionally forgiving:

• Minimum Budget Setup: An NVIDIA GeForce RTX 3060 (12GB) or RTX 4060 Ti (16GB). These cards run the Q5_K_M or BF16 weights with medium-to-long context windows entirely in VRAM.
• The Best GPU for NVIDIA-Nemotron-Labs-Dory1.0-4B: An NVIDIA GeForce RTX 4070 (12GB) or RTX 4080 (16GB). The Ada Lovelace architecture natively accelerates the NVFP4 format, yielding over double the execution speed of BF16.
• Apple Silicon: Macs with unified memory (M2, M3, or M4 with 16GB or 24GB memory) handle GGUF builds effortlessly through Metal, though they process recurrent loops using standard CPU/GPU unified pathways rather than native NVFP4 tensor cores.

NVIDIA-Nemotron-Labs-Dory1.0-4B tokens per second metrics vary significantly based on your configured loop count:

• 1 Loop (Fast Mode): Delivers roughly 90 to 130 tokens per second on an RTX 4090 or M4 Max. Best for general conversation and basic formatting.
• 2 Loops (Recommended Default): Operates at roughly 55 to 80 tokens per second on an RTX 4090. This configuration strikes the optimal balance between coding accuracy and throughput.
• 4 Loops (Deep Reasoning): Yields roughly 28 to 45 tokens per second on an RTX 4090. While generation speed drops proportionately to the extra recurrent cycles, the output arrives without the verbosity of multi-thousand-token chain-of-thought output.

Support for recurrent looped architectures is available via llama.cpp and vLLM through dedicated pull requests and builds. Ollama provides the fastest local ingestion once the GGUF model files are imported.

To pull and run via GGUF in llama.cpp:

1# Clone and build llama.cpp with looped model support
2git clone https://github.com/ggerganov/llama.cpp
3cd llama.cpp && make LLAMA_CUDA=1
4
5# Run inference specifying 2 recurrent loops
6./llama-cli \
7  -m ./models/nemotron-labs-dory1.0-4b-q5_k_m.gguf \
8  -c 32768 \
9  --n-gpu-layers 99 \
10  --loops 2 \
11  -p "Write an asynchronous Python worker using Redis Streams."

In production pipelines, vLLM provides continuous batching with NVFP4 support on Ada GPUs, maximizing total server throughput.

Evaluating NVIDIA-Nemotron-Labs-Dory1.0-4B performance against its peers highlights a clear architectural divergence in the 3B-9B local class.

• NVIDIA-Nemotron-Labs-Dory1.0-4B vs. Qwen2.5-Coder-7B: Qwen2.5-Coder-7B has been a developer favorite for local programming. At 1 loop, Dory performs slightly below Qwen on complex coding benches. At 4 loops, however, Dory matches or exceeds Qwen2.5-Coder-7B on SWE-bench Verified while requiring half the physical parameter footprint in VRAM. If your GPU has only 8 GB of memory, Dory is the superior option because it leaves sufficient headroom for long-context KV caching.
• NVIDIA-Nemotron-Labs-Dory1.0-4B vs. Phi-3.5-mini (3.8B): Microsoft Phi-3.5-mini is a standard dense model with strong reasoning for its size, but its architecture is fixed at inference time. Dory1.0-4B outclasses Phi-3.5 on agentic coding and terminal tasks when loop count is raised to 2 or 4. Phi-3.5 maintains a slight speed advantage if Dory is locked to high loop counts, but Dory offers far greater flexibility for varying compute budgets.

Choose Dory1.0-4B if you need a memory-efficient local model capable of scaling up its reasoning depth for complex coding agents without consuming the 16 GB to 24 GB of VRAM normally demanded by 9B-14B architectures.