harrier-oss-v1-27b

Microsoft’s harrier-oss-v1-27b is a 27-billion-parameter decoder-only multilingual embedding model that topped the Multilingual MTEB v2 leaderboard at release with a score of 74.3. It is the largest variant in the open-source Harrier family, built from fine-tuned Gemma and Qwen architectures. Unlike most embedding models that rely on encoder-only designs (e.g., BERT-based), Harrier uses a causal decoder—the same backbone as many modern LLMs—adapted with last-token pooling and L2 normalization to produce dense text vectors.

This model fills a specific gap: high-quality, open-weight multilingual embeddings that can process extremely long inputs (up to 32,768 tokens) without aggressive chunking. It competes directly with proprietary offerings like zembed-1 and voyage-4, as well as other open models at similar scale (e.g., BGE-M3 and multilingual E5 variants). For teams building retrieval-augmented generation (RAG) pipelines, cross-lingual search, or enterprise document classification, harrier-oss-v1-27b offers a state-of-the-art baseline under an MIT license, meaning no restrictions on commercial use or deployment.

Harrier-oss-v1-27b is a dense decoder-only model—no mixture-of-experts (MoE). Every forward pass activates all 27 billion parameters, making it compute-heavy relative to MoE models of similar total parameter count. However, dense architectures often produce more predictable embeddings and simpler quantization behavior, which matters for production reliability.

Key specifications:

• Parameters: 27B (dense)
• Embedding dimension: 5,376
• Maximum input tokens: 32,768
• Pooling: Last-token pooling
• Normalization: L2-normalized embeddings
• Training objective: Contrastive learning on a large multilingual dataset mixture; larger variants (0.6B and 27B) trained without knowledge distillation from smaller teachers (only the 270M and 0.6B used distillation).

The 32k context length is a standout feature. Most embedding models cap at 512 or 8,192 tokens. With 32,768 tokens, you can embed entire legal contracts, research papers, or codebases in a single pass, eliminating the need for document chunking and the associated retrieval fragility. However, that long context also increases memory usage—a tradeoff to plan for during local deployment.

The model uses last_token_pooling rather than mean pooling. Because the decoder is causal, the last token’s hidden state encapsulates the full sequence representation. This design simplifies inference and is well-supported by the sentence-transformers library and Transformers pipelines.

Harrier-oss-v1-27b is a multilingual text embedding engine optimized for retrieval, clustering, semantic similarity, classification, bitext mining, and reranking. It supports over 100 languages—from English and Chinese to Amharic, Mongolian, and Yoruba. This makes it a strong candidate for global enterprise search, cross-lingual RAG, and document deduplication across language boundaries.

Concrete use cases:

• Enterprise search: Index technical documentation, support tickets, and internal knowledge bases in multiple languages with a single model. The 32k context allows indexing full reports without splitting.
• Legal and contract workflows: Embed entire agreements for clause similarity search, risk classification, or clause extraction.
• Cross-lingual retrieval: Query in English and retrieve relevant documents in Spanish, Arabic, or Japanese with high accuracy.
• RAG pipelines: Use harrier-oss-v1-27b as the embedding backbone for a local RAG system. Its high MTEB scores translate directly to improved retrieval recall.
• Bitext mining: Align parallel sentences across languages for translation memory or training data.

In head-to-head evaluations on 28 graded datasets (using continuous relevance scores), harrier-oss-v1-27b outperforms voyage-4 on average NDCG@10 (0.706 vs 0.702) but trails zembed-1 (0.715). For tasks that require very fine-grained relevance ranking, the model performs best on datasets with long-form queries and documents.

This is a large model. Running it locally requires serious hardware, but it is feasible with modern consumer GPUs using quantization.

• Full precision (FP32): ~108 GB—impractical for most local setups.
• FP16 / BF16: ~54 GB VRAM. An NVIDIA A100 (80 GB) or dual RTX 6000 Ada (48 GB each) can handle this, but not a single consumer card.
• Q4_K_M (4-bit GGUF): ~16 GB VRAM. This is the sweet spot for single-GPU inference on an RTX 4090 (24 GB) or RTX 3090 (24 GB). With 16 GB used for model weights, you have headroom for context and overhead.
• Q3_K_M (3-bit): ~12 GB, allowing use on an RTX 3080 (12 GB) or RTX 4060 Ti 16 GB—but expect some quality degradation on MTEB scores (likely 1–2 points lower).
• Q2_K (2-bit): ~9 GB, borderline on an RTX 3060 12 GB—not recommended for production.

With Q4_K_M on an RTX 4090, expect:

• Tokens per second: ~40–70 tokens/s for embedding generation (batch size 1). Batch encoding multiple sequences improves throughput.
• Time to embed a 1,000-word document: ~0.5–1 second.
• Memory usage: ~16–18 GB VRAM with a context length of 8,192 tokens; at full 32,768 tokens, you’ll need the entire 24 GB.

1. Hardware: NVIDIA RTX 4090 (24 GB) or RTX 3090 (24 GB). For smaller budgets, an RTX 3080 12 GB can run Q3_K_M quantizations.
2. Software: Use Ollama (the model is available in the Ollama library) for the fastest path. Otherwise, load with llama.cpp or sentence-transformers with model_kwargs={"dtype": "auto"} (supports FP16 if you have the VRAM).
3. Quantization: Start with Q4_K_M. It offers a good balance of quality (near FP16) and memory savings. If you need to fit on 12 GB GPUs, try Q3_K_M.
4. Context handling: The model supports 32,768 tokens, but you don’t always need it. For typical retrieval, 8,192 tokens is often sufficient and saves VRAM.

To run with Ollama: ollama run microsoft/harrier-oss-v1-27b:q4_K_M

• zembed-1 holds the #1 spot on MTEB with an average NDCG@10 of 0.715 vs. harrier’s 0.706. On dataset-level matchups, zembed-1 wins 17 of 28 datasets.
• Tradeoff: harrier-oss-v1-27b is open-weight and MIT licensed; you can deploy it anywhere without API costs or data send-off. If you need the highest possible ranking accuracy and can afford a proprietary API, zembed-1 is better. For self-hosted, privacy-sensitive workflows, harrier is the clear choice.

• voyage-4 and harrier-oss-v1-27b are nearly tied on overall averages. On NDCG@10, harrier edges ahead by ~0.004. Both are excellent for general-purpose multilingual retrieval.
• Tradeoff: voyage-4 may have better support for specific enterprise verticals (e.g., code search, finance) but requires an API key. Harrier offers full control, no rate limits, and the ability to fine-tune further.

• BGE-M3 is much smaller and runs on consumer hardware easily (e.g., 4 GB VRAM). It scores ~60 on Multilingual MTEB v2.
• Tradeoff: harrier-oss-v1-27b dramatically outperforms BGE-M3 on every metric, especially cross-lingual and long-context tasks. You pay for that with hardware requirements. Choose harrier if retrieval accuracy is critical and you have the GPU budget; pick BGE-M3 for lightweight or edge deployments.

For most practitioners building local RAG systems with a 24 GB GPU, harrier-oss-v1-27b in Q4_K_M is the strongest open embedding model available today.