SFR-Embedding-Mistral

Our Take

A solid 7.1B-parameter dense embedding model from Salesforce. Treat the modality benchmarks above as the leading indicator of fit — composite scoring across modalities is still maturing.

Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.

SFR-Embedding-Mistral is a dense transformer with 7.1B parameters — the same architecture as Mistral-7B, but fine-tuned for embedding tasks. It uses last-token pooling to generate embeddings from the final hidden state.

Key architectural choices:

• Base model: Mistral-7B-v0.1 (a decoder-only dense model)
• Fine-tuning method: LoRA adapters (rank 8) added to all linear layers, resulting in only 21M trainable parameters
• Training data: Multi-task mixture including retrieval (QA, fact-checking), clustering (scientific/medical articles), classification (reviews, intent, toxic detection), STS, and reranking
• Sequence length: 128 tokens for queries, 256 tokens for documents during training. The base Mistral model supports up to 4096 tokens, so you can use longer sequences in practice.
• Batch construction: Task-homogeneous batching — batches contain samples from a single task, making in-batch negatives more challenging. This contributed a 0.8 point boost on retrieval.

The model uses contrastive loss with hard negatives (7 per query-document pair). It was trained on 8 A100 GPUs for about 15 hours. For inference, you get a single dense vector per input text, normalized for cosine similarity.

SFR-Embedding-Mistral is not a general-purpose language model — it produces text embeddings. Its strength lies in:

• Information retrieval: Given a query, find the most relevant documents. It achieves 59.0 nDCG@10 on MTEB retrieval tasks.
• Semantic textual similarity (STS): Measure how similar two texts are. Scores above 89 on STS benchmarks.
• Clustering: Group documents by topic or theme. +1.4 improvement over E5-mistral on clustering.
• Classification: Use embeddings as features for downstream classifiers. 95.9% accuracy on Amazon polarity classification.
• Reranking: Fine-tuned on scientific and technical forum data for second-stage ranking.

Concrete use cases:

• Local RAG: Embed your knowledge base (PDFs, wikis, codebases) and search with natural language queries.
• Document deduplication: Compare embedding cosine similarity to find near-duplicates.
• Zero-shot classification: Compute similarity between a text and candidate labels.
• Semantic search for internal tools: Replace keyword search with embedding-based retrieval on your own hardware.

This is a 7.1B dense model — it will run on consumer GPUs with 8GB+ VRAM, but performance varies by quantization.

• Minimum: Any GPU with 8GB VRAM (RTX 3070, RTX 4060, M4 Pro) — use Q4_K_M quantization.
• Recommended: RTX 4090 (24GB) or M4 Max (40+GB unified memory) — run FP16 for full precision.
• CPU only: Possible with llama.cpp and 4-bit quantization, but expect 1–2 tokens per second for embedding generation. Not recommended for production.

Embedding generation is fast because it’s a single forward pass per text. On an RTX 4090 with FP16, you can expect ~200–400 tokens per second (batch size 1). On an RTX 3070 with Q4_K_M, expect ~80–150 tokens per second. Throughput scales with batch size — you can embed thousands of documents in seconds.

1. Ollama (easiest): ollama run nomic-embed-text is a smaller alternative, but for SFR-Embedding-Mistral you can use llama.cpp directly.
2. llama.cpp / llama-server: Download the GGUF quantized model from HuggingFace (e.g., SFR-Embedding-Mistral-Q4_K_M.gguf) and run:

1. Sentence Transformers (Python): Load the model with sentence-transformers library — works with CUDA, but requires full precision (14GB VRAM).

For best quantization, Q4_K_M is the sweet spot for most users — minimal quality loss at half the VRAM. If you have 24GB, Q5_K_M or FP16 gives slightly better retrieval accuracy.

SFR-Embedding-Mistral vs E5-mistral-7b-instruct: Both are based on Mistral-7B. SFR has better retrieval (+2.1 points) and clustering (+1.4) thanks to multi-task training and hard negatives. E5 is MIT-licensed, so it’s the choice for commercial use.

SFR-Embedding-Mistral vs GTE-Qwen2-7B: GTE-Qwen2-7B (by Alibaba) scores slightly higher on MTEB overall (~68) and is MIT-licensed. SFR still leads on retrieval (59.0 vs ~58.5). If you need commercial use, choose GTE. If you want the best retrieval for research, SFR is still competitive.

When to pick SFR-Embedding-Mistral: You’re doing research or non-commercial RAG, need top-tier retrieval accuracy, and have the VRAM to run a 7B model. If you need commercial license or lower VRAM, look at BGE-M3 (567M, fits 4GB GPUs) or E5-mistral-7b-instruct.