BidirLM-1B-Embedding

Our Take

A workable 1B-parameter dense embedding model from BidirLM. Treat the modality benchmarks above as the leading indicator of fit — composite scoring across modalities is still maturing.

Generated from this model’s benchmarks and ranking signals. Editor reviews refine it over time.

• Parameters: 1,001M (dense)
• Architecture: Dense transformer with full bidirectional attention
• Embedding Dimension: 1152
• Context Length: 512 tokens for MTEB evaluation; underlying Gemma3 backbone supports up to 32,768 tokens (adjust model.max_seq_length or max_length accordingly)
• Modality: Text‑only
• Languages: Multilingual — over 140 languages supported natively by the Gemma3 base, reinforced with contrastive training covering 87 languages

The adaptation process is critical: unlike contrastive‑only models, BidirLM first trains a Fill‑Mask checkpoint via MNTP, which teaches the causal LLM to attend bidirectionally. This step prevents catastrophic forgetting and makes the encoder effective for token‑level tasks (NER, classification) as well as generic embedding benchmarks. The final embedding model (BidirLM-1B-Embedding) is the MNTP checkpoint further tuned with contrastive losses.

Because it’s a dense model, every parameter is active during inference. This means VRAM consumption scales linearly with parameter count: a 1B model at FP16 requires roughly 2 GB of GPU memory, making it far more memory‑efficient than MoE variants of the same size.

BidirLM-1B-Embedding excels at two broad categories of work: generic text embeddings (via Sentence Transformers) and downstream fine‑tuning (via HuggingFace Transformers).

• Semantic Textual Similarity (STS) – producing high‑quality cosine similarities for sentence pairs
• Dense Retrieval – encoding queries and documents for retrieval‑augmented generation or search
• Clustering & Classification – sentence‑level tasks where a single vector per input suffices
• Reranking & Bitext Mining – pair‑wise scoring and cross‑lingual alignment
• Multilabel Classification – predicting multiple labels from a single representation

Because the model was pre‑trained with MNTP, it retains strong token‑level understanding. You can fine‑tune it directly for:

• Sequence classification (e.g., MNLI, XNLI, PAWS‑X)
• Token classification (e.g., NER with PAN‑X, POS tagging)
• Information retrieval (e.g., MIRACL, CodeSearchNet)
• Sequence regression (e.g., Seahorse for quality estimation)

Concrete use cases: building a multilingual semantic search engine for 100+ languages, fine‑tuning a custom NER pipeline on legal documents, or replacing a heavier embedding model in a retrieval‑augmented pipeline on a single RTX 4090.

This is where the model shines: you can run it on a single consumer GPU without compromises.

• FP16 (full precision): ~2 GB – fits on any modern GPU with 4 GB+ (GTX 1060 6GB, RTX 3050)
• Q4_K_M (4‑bit quantization): ~1.2 GB – runs on integrated graphics or 2GB VRAM cards
• Q8_0 (8‑bit): ~1.6 GB
• Recommended: Q4_K_M gives negligible quality loss vs FP16 while cutting memory by 40%. Start there.

• RTX 3070 / 4070: FP16 comfortably, batch size 8+ for retrieval
• RTX 4090: Overkill – batch size 64+ at FP16, hundreds of tokens per second
• Apple M4 Max (64GB unified): FP16 with room to spare; M1/M2 with 16GB also viable at Q4_K_M
• Laptop GPUs (RTX 3050, 4GB): Q4_K_M is your friend – expect 20–40 tokens/second for single‑sequence encoding

Tokens‑per‑second depends on hardware and quantization. Rough estimates for a single sequence (512 tokens):

• RTX 4090 (Q4_K_M): 500–700 t/s
• RTX 3070 (Q4_K_M): 200–350 t/s
• M4 Max (Q4_K_M): 300–450 t/s
• RTX 4090 (FP16): 250–400 t/s

Processing documents in batches amplifies throughput linearly up to VRAM capacity.

The easiest way to run it locally is via [Ollama](https://ollama.com). While an official model may not be available on day one, you can pull the GGUF conversion from HuggingFace community repos or convert the model yourself with llama.cpp. Alternatively, use Sentence Transformers directly:

Note the trust_remote_code=True requirement – this is mandatory because BidirLM uses custom modeling code to enable bidirectional attention.

When to choose BidirLM‑1B: If you need the highest multilingual embedding quality at the 1B scale, especially for languages beyond English. The MNTP pre‑training also makes it uniquely suited for fine‑tuning on token‑level tasks – BGE‑M3 and Stella are contrastive‑only and lack this capability. The 32K token ceiling (via Gemma3 backbone) also allows longer documents when you increase max_seq_length, though MTEB scores are only validated at 512.

When to choose an alternative: If you’re constrained to a sub‑500M model (e.g., edge devices), Stella‑400M or BGE‑Small offer lower VRAM. For English‑only retrieval at higher throughput, BGE‑M3 is slightly faster and permissively licensed. BidirLM’s Gemma License imposes usage restrictions (see the license for details) – if Apache‑2.0 or MIT is mandatory, the alternatives are safer.