Tencent Hy4 preview

Tencent Hy4 preview is an open-weight, 770-billion-parameter Mixture-of-Experts (MoE) flagship model engineered for enterprise reasoning, long-horizon software engineering, and document-scale data synthesis. Released on August 28, 2026, it represents one of the largest open-weights releases available, directly challenging closed-source commercial APIs and open frontier models like DeepSeek-V3 and Llama 3.1 405B.

The key to this architecture is dynamic sparsity: while the model maintains a massive 770B parameter footprint in storage and memory, it activates only 49B parameters per token during inference. This design allows it to deliver the depth and generalization of a near-trillion-parameter system while maintaining the per-token computational cost of a mid-sized model. Combined with a native 1,000,000-token context window, Hy4 preview targets complex workflows where typical dense models fail, including full-repository code refactoring, complex legal and financial reconciliation, and scientific data extraction.

For engineers seeking to run Tencent Hy4 preview locally, the system demands serious planning around memory capacity and interconnect bandwidth. Staging a 770B model requires enterprise multi-GPU nodes or high-capacity unified memory workstations. However, for organizations that require complete data sovereignty for internal codebases or proprietary financial models, it offers true frontier-tier execution entirely on-premise.

Hy4 preview uses a 78-layer transformer architecture optimized for memory efficiency and token throughput. Rather than applying MoE uniformly, the backbone utilizes a dense Feed-Forward Network (FFN) for its initial layer, followed by 77 MoE layers.

1Total Parameters:           770 Billion
2Active Parameters / Token:  49 Billion
3Backbone Layers:            78 (1 Dense FFN + 77 MoE)
4Routed Experts per Layer:   256 (Top-8 routed per token)
5Shared Experts per Layer:   1 (Always active)
6Attention Mechanism:        Gated DeepSeek Sparse Attention (DSA) with IndexCache
7Residual Connections:       Independent Hyper-Connections (iHC, 4 streams)
8Context Window:             1,000,000 tokens
9Speculative Decoding:       Native Multi-Token Prediction (MTP) layer (10B total / 0.7B active)

Each MoE layer combines 256 routed experts with one dedicated, always-active shared expert. Every token routes to the top 8 routed experts in addition to the shared expert, bringing the active compute budget to 49B parameters per forward pass. The inclusion of an integrated 10B-parameter Multi-Token Prediction (MTP) layer enables native speculative decoding, allowing supported runtimes to predict multiple tokens per step without requiring an external draft model.

Memory management at 1M context is stabilized through Gated DeepSeek Sparse Attention (DSA) paired with Multi-head Latent Attention (MLA). The model compresses Key-Value (KV) representations into a low-rank latent space (512-dimension KV compression) before projection. It also employs IndexCache to share sparse index lookups across adjacent layers, cutting down the memory growth of the KV cache over hundreds of thousands of tokens. To maintain stability across its 78-layer depth, the architecture replaces standard residual connections with Independent Hyper-Connections (iHC), routing four parallel information streams through each block to prevent representation collapse.

Tencent Hy4 preview MoE efficiency relies entirely on fast memory access. While compute requirements scale with the 49B active parameters, the hardware must hold all 770B parameters in accessible memory to avoid severe routing stalls during expert selection.

Hy4 preview is built around heavy, document-dense workflows and extended reasoning chains.

• Long-Horizon Software Engineering: Tencent Hy4 preview for coding goes beyond single-function generation. With a 1M-token window, it can digest complete dependency graphs, internal documentation, and legacy codebases simultaneously. It excels at multi-file refactoring, locating architectural anti-patterns, generating front-end code with precise visual design specifications (such as WebGL and Three.js implementations), and validating automated test suites.
• Financial and Document Analysis: The model handles unstructured, multi-document auditing where policies, balance sheets, and email records conflict. It tracks context across hundreds of pages to cross-verify claims, detect accounting discrepancies, and output structured reconciliations with explicit citations.
• Scientific and Technical Reasoning: Across reasoning benchmarks, the model demonstrates high stability in mathematical proofs, algorithmic design, and multi-step logic. It parses complex equations and dense technical literature without suffering attention drift over long contexts.
• Instruction-Following and Artifact Generation: It reliably adheres to rigid schema constraints, outputting syntactically valid JSON, SQL pipelines, and custom reporting tables without injecting conversational filler.

Deploying a local AI model with 770B parameters in 2026 is an infrastructure-level undertaking. The primary bottleneck is not floating-point compute, but the total VRAM capacity required to store the weights and the memory bandwidth needed to stream them.

The table below breaks down the memory required to host the model weights alongside a realistic KV cache allocation.

If you are wondering how to run 770B model on consumer GPU setups, the short answer is: you cannot run it efficiently on single or typical dual-GPU workstations.

Even at a compressed 4-bit quantization, the model requires roughly 440 GB of addressable space before allocating a single token of KV cache. A system loaded with multiple 24GB GPUs (like the RTX 4090) would require at least 20 to 24 cards connected over PCIe, where inter-card latency severely chokes MoE expert routing.

For individual practitioners and small labs, the most viable path to run Tencent Hy4 preview locally without an enterprise cluster is high-density unified memory. Workstations running multiple Apple Silicon systems or dual-socket AMD EPYC servers with 768GB to 1TB of DDR5 system RAM can host the model via llama.cpp or Ollama. However, DDR5 system memory bandwidth caps execution speed to roughly 0.5 to 2.5 tokens per second.

For interactive speeds, the best GPU for Tencent Hy4 preview is an enterprise-grade cluster: an 8x NVIDIA H100 (80GB), H200 (141GB), or RTX 6000 Ada (48GB) node linked via high-speed NVLink.

The best quantization for Tencent Hy4 preview in production is Q4_K_M (or 4-bit AWQ/GPTQ). MoE architectures preserve reasoning quality exceptionally well at 4-bit precision because the routing gate maintains dynamic range, while the memory footprint drops by more than 70% compared to native 16-bit weights.

Expected generation speeds vary widely based on your hardware interconnect:

• 8x NVIDIA H100 SXM (80GB) via vLLM / SGLang (4-bit): 25 to 45 tokens per second.
• 8x RTX 6000 Ada (48GB) via PCIe (4-bit): 10 to 18 tokens per second.
• Clustered Consumer GPUs (16x RTX 4090 via PCIe): 3 to 7 tokens per second (heavily latency-bound).
• Server CPU / Unified RAM (1TB DDR5): 0.8 to 2.2 tokens per second.

For production multi-GPU environments, deploy Hy4 preview using vLLM or SGLang with tensor and pipeline parallelism:

1# Serving via vLLM across an 8-GPU node
2vLLM serve tencent/Hy4-preview \
3    --tensor-parallel-size 8 \
4    --quantization fp8 \
5    --max-model-len 65536 \
6    --trust-remote-code

For testing and development on high-RAM workstations, Ollama provides the simplest local setup:

1# Pull and run the 4-bit quantized build via Ollama
2ollama run tencent-hy4:q4_k_m

Ensure your swap space is disabled and transparent huge pages are enabled to prevent memory stalls when staging the 440GB+ checkpoint into RAM.

To determine whether Hy4 preview fits your infrastructure, evaluate it against comparable high-parameter models:

1Model                 Parameters (Total / Active)   Context Window   Minimum 4-Bit Memory
2Tencent Hy4 preview   770B / 49B (MoE)              1,000,000        ~480 GB
3DeepSeek-V3           671B / 37B (MoE)              128,000          ~410 GB
4Meta Llama 3.1 405B   405B / 405B (Dense)           128,000          ~250 GB

Both models employ multi-head latent attention, dynamic MoE routing, and native speculative decoding strategies. DeepSeek-V3 activates fewer parameters per token (37B vs 49B) and has a smaller overall footprint (671B vs 770B), making it slightly easier to squeeze onto an 8x 80GB GPU cluster with full KV cache allocation. However, Hy4 preview offers a native 1-million-token context window compared to DeepSeek-V3's 128k baseline. If your workload involves ingesting massive document bundles or enterprise-wide code repositories, Hy4 preview provides a substantial functional advantage.

Llama 3.1 405B is a dense architecture: every forward pass engages all 405 billion parameters. Consequently, Llama 3.1 405B requires less raw VRAM to stage its weights (~250 GB at 4-bit compared to ~440 GB for Hy4 preview), meaning it can run on smaller multi-GPU nodes. However, Hy4 preview's active parameter cost is only 49B. Once the weights are successfully held in memory, Hy4 preview delivers significantly higher tokens-per-second throughput and requires far less floating-point compute per generated token than Llama 3.1 405B.