Hy3

Hy3 is Tencent’s open-weight Mixture-of-Experts language model, formerly branded as Hunyuan 3.0. With 295 billion total parameters and only 21 billion active per token, Hy3 occupies a distinct niche: it delivers reasoning and agent performance comparable to dense models two to five times its size, while keeping inference costs and memory demands closer to a 20B-parameter model. Released under Apache 2.0, Hy3 targets developers who need a capable, locally runnable model for coding, math, complex reasoning, and function-calling workflows — without tying themselves to a cloud API.

Tencent positions Hy3 as a hybrid fast-and-slow-thinking model. It uses 192 experts with top-8 routing, a 256K-token context window, and a multi-token prediction (MTP) layer. The model ships with documented benchmarks: 90.4 on GPQA Diamond, 78 on SWE-bench Verified, 57.9 on SWE-bench Pro, and 53.2 on Humanity’s Last Exam. These numbers place it alongside or above much larger open-weight models in reasoning and software engineering tasks.

Hy3 is an MoE (Mixture of Experts) transformer with 80 decoder layers (excluding the MTP layer), 64 attention heads using Grouped Query Attention (GQA, 8 KV heads, head dim 128), hidden size 4096, and intermediate size 13312. The total parameter count is 295B, but the key number for anyone running it locally is the 21B active parameters per token. The vocabulary size is 120,832, and the model supports BF16 precision natively.

What “active parameters” means in practice: during inference, only 8 out of 192 experts are activated per token. This keeps the compute footprint roughly equivalent to that of a 21B-parameter dense model, while the remaining 274B parameters sit in memory as static weights. The memory requirement is still dominated by the total parameter count because all expert weights must be loaded into VRAM (or offloaded via CPU RAM). For a 295B model in BF16, that’s roughly 590 GB of VRAM for the full model. Quantization is essential for consumer hardware.

The context window of 256K tokens is a standout feature. It enables processing entire codebases, long technical documents, or multi-turn agent conversations in a single pass. The MTP layer (3.8B parameters) is a separate prediction head that can be disabled during inference if not needed, saving memory.

Hy3 is a text-only model with demonstrated strengths in:

• Reasoning: Complex math, logic, STEM problem-solving. The GPQA Diamond score (90.4) is competitive with models like GPT-4 and Claude 3.5 on graduate-level science questions.
• Coding: SWE-bench Verified (78) and SWE-bench Pro (57.9) indicate strong performance on real-world software engineering tasks — bug fixing, code generation, test generation. Tencent’s internal blind evaluation with 270 experts rated Hy3 at 2.67/4, outperforming GLM-5.1 (2.51/4), especially in frontend development, data & storage, and CI/CD.
• Agent workflows: Function-calling, multi-step planning, tool use. The model was refined through feedback from 50+ Tencent products (WorkBuddy, CodeBuddy, Yuanbao, Marvis, ima) and is designed for autonomous agent loops.
• Math: Strong on formal and informal math benchmarks, including PhD-level qualifying exams.
• Multilingual: While the model card doesn’t specify language coverage, the vocabulary size and Tencent’s product ecosystem suggest robust Chinese and English support, with likely coverage of other major languages.
• Instruction-following: The release notes emphasize improvements in instruction adherence and in-context learning over the preview version.

Concrete use cases: a developer running Hy3 locally can use it as a coding assistant that understands entire repositories (thanks to 256K context), an agent backend for automated bug fixing, a math tutor for advanced problems, or a drop-in replacement for cloud-based reasoning models in cost-sensitive workflows.

Hy3 is a 295B parameter model. Running it on consumer hardware requires aggressive quantization and possibly CPU offloading. Here are realistic scenarios:

• Q4_K_M (4-bit, ~150 GB): This is the most practical balance for a single high-end GPU with 48 GB VRAM — but you’ll still need to offload layers to system RAM. With a fast CPU and enough RAM (128 GB+), you can get 1–2 tokens/second. Tools like llama.cpp support this.
• Q2_K (2-bit, ~75 GB): Fits on a single 80 GB GPU (e.g., NVIDIA A100 or H100), but not on consumer cards. On a 48 GB GPU, you’ll need offloading. Speed drops significantly.
• Q8_0 (8-bit, ~300 GB): Requires multiple GPUs (e.g., 4x RTX 4090s or 2x A100s). Not practical for most local setups.

• Minimum: 48 GB VRAM (e.g., RTX 4090, RTX 6000 Ada) + 128 GB system RAM + fast CPU (e.g., AMD Ryzen 7950X or Intel i9-13900K). Expect 0.5–1.5 tokens/sec with Q4_K_M and CPU offloading.
• Recommended: 2x RTX 4090 (48 GB each) or 1x A100 80 GB for Q4_K_M without offloading, yielding 5–10 tokens/sec.
• Best: 4x RTX 4090 or 2x A100 80 GB for Q8 or BF16, but that’s a server-grade setup.

• Ollama: The quickest start. Hy3 is likely available in the Ollama library (or you can import a GGUF conversion). Use ollama run hy3:q4_K_M to get started.
• vLLM: Recommended for production deployments. Tencent provides official vLLM and SGLang support on the Hugging Face model card.
• llama.cpp: For custom quantization and CPU-offloading setups.

• Q4_K_M on a single RTX 4090 with CPU offloading: ~1 token/sec.
• Q4_K_M on two RTX 4090s: ~5–8 tokens/sec.
• BF16 on a server with 8x A100: >100 tokens/sec.

The MoE architecture means that even with only 21B active parameters, the model’s memory bandwidth bottleneck is the total parameter count. Quantization is the only way to make it viable on consumer hardware, and even then, expect slow speeds. Hy3 is best suited for batch or offline reasoning tasks, not real-time chat, on typical local setups.

DeepSeek-V3 is another MoE model with a similar philosophy — larger total parameters but more active per token. Hy3’s 21B active vs. DeepSeek-V3’s 37B active means Hy3 is compute-cheaper per token but has less capacity per forward pass. In practice, Hy3’s scores on GPQA Diamond (90.4) and SWE-bench Verified (78) are competitive with or exceed DeepSeek-V3’s reported numbers, while requiring less VRAM for the same quantization. Choose Hy3 if you want a smaller active footprint and a more permissive license (Apache 2.0 vs. DeepSeek’s custom license). Choose DeepSeek-V3 if you need the extra capacity for very long reasoning chains or if you have the hardware to support its larger active size.

Qwen2.5-72B is a dense model that requires ~140 GB in BF16. Hy3 quantized to Q4 (~150 GB) has similar memory requirements but offers 256K context vs. Qwen’s 128K, and significantly better reasoning benchmarks (90.4 vs. ~85 on GPQA Diamond). However, Qwen2.5-72B runs faster on a single GPU because it’s a dense forward pass with no expert routing overhead. If you have a 48 GB GPU and can’t offload, Qwen2.5-72B at Q4 is a better fit. If you have two GPUs or can tolerate CPU offloading for the sake of higher intelligence, Hy3 wins.

• You need the best open-weight reasoning and coding performance under Apache 2.0.
• You have multiple GPUs or a server-class setup and can tolerate slower inference for higher accuracy.
• You require a 256K context window for long-code or multi-document tasks.
• You want to build agent workflows with function-calling and tool integration.

• You’re limited to a single consumer GPU with 24 GB VRAM — even quantized Hy3 won’t fit comfortably. Consider smaller models like Qwen2.5-32B or Llama 3.1-70B.
• You need real-time streaming responses — Hy3’s MoE routing overhead and large total size make it slower than similarly quantized dense models.