SFR-Embedding-Mistral

SFR-Embedding-Mistral is a 7.1 billion parameter text embedding model from Salesforce Research. It was the first model to hit the top of the Massive Text Embedding Benchmark (MTEB) leaderboard in February 2024, achieving an average score of 67.6 across 56 datasets. This is a dense, contrastive embedding model built on top of E5-mistral-7b-instruct and Mistral-7B-v0.1.

What sets it apart is its retrieval performance: it jumped from a score of 56.9 (E5-mistral-7b-instruct) to 59.0 on the MTEB retrieval subset, and showed a +1.4 improvement on clustering tasks. If you need a local embedding model for semantic search, RAG pipelines, or document clustering, this is a strong candidate.

The model is released under the CC-BY-NC-4.0 license, which means it’s free for research and non-commercial use. For commercial applications, you’ll need to check with Salesforce or consider alternatives.

SFR-Embedding-Mistral is a dense transformer with 7.1B parameters — the same architecture as Mistral-7B, but fine-tuned for embedding tasks. It uses last-token pooling to generate embeddings from the final hidden state.

Key architectural choices:

• Base model: Mistral-7B-v0.1 (a decoder-only dense model)
• Fine-tuning method: LoRA adapters (rank 8) added to all linear layers, resulting in only 21M trainable parameters
• Training data: Multi-task mixture including retrieval (QA, fact-checking), clustering (scientific/medical articles), classification (reviews, intent, toxic detection), STS, and reranking
• Sequence length: 128 tokens for queries, 256 tokens for documents during training. The base Mistral model supports up to 4096 tokens, so you can use longer sequences in practice.
• Batch construction: Task-homogeneous batching — batches contain samples from a single task, making in-batch negatives more challenging. This contributed a 0.8 point boost on retrieval.

The model uses contrastive loss with hard negatives (7 per query-document pair). It was trained on 8 A100 GPUs for about 15 hours. For inference, you get a single dense vector per input text, normalized for cosine similarity.

SFR-Embedding-Mistral is not a general-purpose language model — it produces text embeddings. Its strength lies in:

• Information retrieval: Given a query, find the most relevant documents. It achieves 59.0 nDCG@10 on MTEB retrieval tasks.
• Semantic textual similarity (STS): Measure how similar two texts are. Scores above 89 on STS benchmarks.
• Clustering: Group documents by topic or theme. +1.4 improvement over E5-mistral on clustering.
• Classification: Use embeddings as features for downstream classifiers. 95.9% accuracy on Amazon polarity classification.
• Reranking: Fine-tuned on scientific and technical forum data for second-stage ranking.

Concrete use cases:

• Local RAG: Embed your knowledge base (PDFs, wikis, codebases) and search with natural language queries.
• Document deduplication: Compare embedding cosine similarity to find near-duplicates.
• Zero-shot classification: Compute similarity between a text and candidate labels.
• Semantic search for internal tools: Replace keyword search with embedding-based retrieval on your own hardware.

This is a 7.1B dense model — it will run on consumer GPUs with 8GB+ VRAM, but performance varies by quantization.

• Minimum: Any GPU with 8GB VRAM (RTX 3070, RTX 4060, M4 Pro) — use Q4_K_M quantization.
• Recommended: RTX 4090 (24GB) or M4 Max (40+GB unified memory) — run FP16 for full precision.
• CPU only: Possible with llama.cpp and 4-bit quantization, but expect 1–2 tokens per second for embedding generation. Not recommended for production.

Embedding generation is fast because it’s a single forward pass per text. On an RTX 4090 with FP16, you can expect ~200–400 tokens per second (batch size 1). On an RTX 3070 with Q4_K_M, expect ~80–150 tokens per second. Throughput scales with batch size — you can embed thousands of documents in seconds.

1. Ollama (easiest): ollama run nomic-embed-text is a smaller alternative, but for SFR-Embedding-Mistral you can use llama.cpp directly.
2. llama.cpp / llama-server: Download the GGUF quantized model from HuggingFace (e.g., SFR-Embedding-Mistral-Q4_K_M.gguf) and run:

1   ./embedding -m SFR-Embedding-Mistral-Q4_K_M.gguf -p "Your text here"

1. Sentence Transformers (Python): Load the model with sentence-transformers library — works with CUDA, but requires full precision (14GB VRAM).

For best quantization, Q4_K_M is the sweet spot for most users — minimal quality loss at half the VRAM. If you have 24GB, Q5_K_M or FP16 gives slightly better retrieval accuracy.

SFR-Embedding-Mistral vs E5-mistral-7b-instruct: Both are based on Mistral-7B. SFR has better retrieval (+2.1 points) and clustering (+1.4) thanks to multi-task training and hard negatives. E5 is MIT-licensed, so it’s the choice for commercial use.

SFR-Embedding-Mistral vs GTE-Qwen2-7B: GTE-Qwen2-7B (by Alibaba) scores slightly higher on MTEB overall (~68) and is MIT-licensed. SFR still leads on retrieval (59.0 vs ~58.5). If you need commercial use, choose GTE. If you want the best retrieval for research, SFR is still competitive.

When to pick SFR-Embedding-Mistral: You’re doing research or non-commercial RAG, need top-tier retrieval accuracy, and have the VRAM to run a 7B model. If you need commercial license or lower VRAM, look at BGE-M3 (567M, fits 4GB GPUs) or E5-mistral-7b-instruct.