AREX-2

AREX-2 is a 27.36B dense multimodal model from BAAI (Beijing Academy of Artificial Intelligence), the lab behind the original AREX deep-research agent family. It occupies a specific niche: long-horizon agent work where the model improves a candidate solution across multiple test-time rounds instead of answering once and stopping. The loop is propose, measure, reflect, revise. Training combines machine-learning and algorithmic-programming tasks with verifiable feedback plus AREX deep-research data, and the self-improvement behavior learned on those tasks transfers to research workloads without adding new search trajectories.

The numbers that matter for evaluation: 70.7 on Frontier-CS, 81.8 on MLE-Lite, 84.0 on BrowseComp, and 92.2 on GAIA. Those are competitive with far larger open-weight models on agentic and ML-engineering benchmarks, and they were produced by a dense 27B checkpoint, not a mixture-of-experts with a large activated-parameter budget.

Why this matters for local deployment: 27.36B dense fits on a single 24GB consumer card at 4-bit quantization, and the weights are open on Hugging Face under Apache 2.0. That combination (agentic tool use, vision input, 262,144-token context, permissive license, single-GPU footprint) is rare in this parameter class. If you are building an agent harness that needs to run on hardware you own, AREX-2 is one of the few serious candidates.

AREX-2 is a dense transformer, not a mixture-of-experts. Every one of the 27.36B parameters is active on every token. Two practical consequences follow:

• VRAM scales with total parameters. There is no activated-parameter discount. You pay for all 27.36B in memory, unlike a 122B-A10B MoE that only loads a fraction per token.
• Throughput is predictable. No routing overhead, no expert imbalance, no per-token variance from gating decisions. Tokens per second stay roughly flat until you hit memory bandwidth limits.

It is built on a Qwen3.8-27B backbone, so it is a Qwen3.8-compatible multimodal model: a vision encoder feeding a text decoder. The Hugging Face pipeline tag is image-text-to-text, and weights ship as safetensors for transformers.

The 262,144-token context window is not a marketing figure here, it is load-bearing. The reflective loop depends on the model reading its own prior state: scores, logs, stack traces, error messages, timings, and the accumulated record of what it already tried. Long context is what lets a single agent session hold an entire ML experiment history or a multi-hour research transcript without an external summarizer.

Declared capabilities are chat, code, vision, reasoning, and function-calling. The realistic use cases follow directly from the training mix.

• Automated ML engineering. Run a training job, parse the logs and metrics, diagnose what failed, revise the config, and repeat. MLE-Lite at 81.8 is the relevant signal. This is the strongest argument for the model.
• Algorithmic programming. Competitive-programming-style problems where a verifier exists and the model can iterate against failing test cases. Frontier-CS at 70.7 puts it in range of much larger closed models.
• Deep research agents. BrowseComp 84.0 and GAIA 92.2. Multi-round search, evidence integration, constraint checking, and answer revision with tool calls.
• Vision-grounded agent work. Screenshots, charts, diagrams, and UI captures as inputs. Useful for browser automation and document extraction pipelines where the model must read a rendered page rather than parsed text.
• Local agent harnesses. Function-calling plus long context plus a permissive license means you can wire it into your own tool loop without a cloud dependency.

Where it is not the right pick: high-volume short-answer chat, broad multilingual work, or anything that needs a generalist's breadth. AREX-2 is specialized.

Weights only, before KV cache:

• BF16 / FP16: roughly 55 GB. Needs an 80GB accelerator or multi-GPU with layer offload.
• Q8_0: roughly 29 GB. Fits a 32GB card (RTX 5090) or 48GB professional cards.
• Q6_K: roughly 23 GB. Tight on a 24GB card with short context, comfortable on 32GB.
• Q4_K_M: roughly 16.5-17 GB. The practical default for 24GB cards.
• Q3_K_M / IQ3: roughly 13 GB. Runs on 16GB cards, with visible degradation on reasoning-heavy tasks.

KV cache is the second budget. At 262,144 tokens the cache is very large (tens of GB depending on attention configuration). For local work, cap context at 32K-64K, which costs a few GB and covers most agent sessions. Full 262K context realistically requires 48GB or more of memory.

• RTX 4090 (24GB): Q4_K_M at 32K-64K context. The most common target.
• RTX 3090 (24GB): same configuration, lower throughput.
• RTX 5090 (32GB): Q6_K or Q8_0 with moderate context.
• M4 Max (64GB or 128GB unified): Q4_K_M or Q6_K with generous context. Generation speed is fine; prompt processing on long transcripts is the weak point.
• 2x 24GB cards: tensor parallelism or layer offload to push into Q6_K/Q8_0 territory.
• CPU-only with 32GB+ RAM: possible at Q4 through llama.cpp, but expect single-digit tokens per second. Not viable for agent loops.

Q4_K_M is the right starting point for most users. Move to Q5_K_M or Q6_K if you have memory headroom, particularly for ML engineering tasks where numeric reasoning and log parsing matter. Q8_0 is worth it only on 32GB+ hardware. Below Q4, the reflective loop degrades first: the model starts misreading its own logs and error output, which defeats the entire point of the architecture.

On an RTX 4090 at Q4_K_M, expect roughly 20-35 tokens/sec on short generations, dropping as context fills. An M4 Max lands nearer 10-20 tokens/sec. Prompt processing is the real bottleneck for long agent transcripts, so prefill speed matters more than raw decode speed for this workload.

Ollama is the fastest path. Check the Ollama library for an AREX-2 tag; if none exists yet, pull a GGUF build from Hugging Face or convert the safetensors yourself with llama.cpp's convert_hf_to_gguf.py. Note that vision requires the matching mmproj file, and not every GGUF release ships one. For multi-user serving or high-throughput batch work, run the native weights under vLLM instead.

AREX-2 vs Qwen3-32B. Qwen3-32B is a dense generalist with strong coding and reasoning and a large ecosystem of quants and tooling. It has no vision in the base checkpoint and no long-horizon self-improvement training. Pick Qwen3-32B for broad chat, multilingual work, and mature local tooling. Pick AREX-2 when the task is iterative with verifiable feedback and you want vision plus 262K context in the same checkpoint.

AREX-2 vs Gemma 3 27B. Gemma 3 27B is dense, multimodal, and runs at a similar memory footprint, but with a 128K context window and a generalist post-training mix. It is the better choice for straightforward vision Q&A and general assistant work. AREX-2 wins on agentic benchmarks, ML engineering, and any workload where the model must sustain productive iteration across many rounds.

The tradeoff is consistent: AREX-2 gives up some generalist breadth for specialization in long-horizon, feedback-driven tasks. If your workload looks like an experiment loop or a research agent, that trade is worth making. If it looks like a chatbot, it is not.