BOOM_4B_v1

BOOM_4B_v1 is a 4 billion parameter dense text embedding model developed by the Institute of Computing Technology (ICT), Chinese Academy of Sciences. It is built on the Qwen3-4B architecture and fine-tuned for general-purpose text representation, with a specific focus on out-of-domain (OOD) robustness. The model uses a novel bagging-based model merging technique (BOOM) that trains multiple embedding models on sampled subsets of training data then merges them into a single model. This approach improves both in-domain and OOD retrieval performance while keeping inference identical to a single model.

The model occupies a specific niche: large-scale text embedders. Most embedding models run under 1B parameters (e.g., BGE-M3, E5, GTE). BOOM_4B_v1 pushes that ceiling to 4B, targeting applications that demand higher representational capacity—particularly in retrieval augmented generation (RAG), enterprise search, and semantic similarity tasks where robustness across unseen domains matters. It competes with other large embedders like intfloat/e5-mistral-7b-instruct (7B) and Alibaba’s gte-Qwen2-1.5B (1.5B), but offers a middle ground in size and a unique training methodology.

BOOM_4B_v1 is a dense transformer, not MoE. All 4B parameters are active during inference. The model is initialized from Qwen3-4B, then trained for sentence embedding using last-token pooling. The base model supports a context length of 32,000 tokens (per the Qwen3-4B specification), enabling encoding of long documents in a single pass.

The key architectural innovation is in training, not in model design. Five embedding models are trained on different random subsets (20%, 40%, 60%, 80%, 100%) of the full 2.8M multi-task corpus. These are then merged using Multi-SLERP (spherical linear interpolation) with weighted coefficients (0.2, 0.4, 0.6, 0.8, 1.0). The resulting model retains the inference cost of a single dense 4B network while capturing the variance-reducing benefits of bagging. This technique avoids the OOD generalization limitations of standard batch-level shuffling and supports incremental updates without full retraining.

The model is distributed in float32 precision. Quantization (e.g., to 8-bit or 4-bit) is supported via common inference engines.

BOOM_4B_v1 is a text-only embedding model designed for:

• Information retrieval / RAG – dense retrieval over large document collections where query-document similarity is computed via cosine similarity on embeddings.
• Semantic textual similarity (STS) – measuring how similar two sentences or paragraphs are.
• Text classification – embedding-based classification using nearest neighbor or linear probes.
• Clustering – grouping documents by semantic content.
• Reranking – rescoring candidate documents from a first-stage retrieval.

Training data covered retrieval (MS MARCO, NQ, HotpotQA, FEVER, FiQA, etc.), reranking (StackOverflowDupQuestions), classification (Amazon Reviews, Banking77, IMDB), clustering (Arxiv, Reddit), STS (STS12-22), and code (Cornstack – JavaScript, Java, Python, PHP, Ruby). The model handles English-dominant text but can encode other languages present in the training mix (e.g., MIRACL, Mr. TyDi). It does not generate text; it produces fixed-length vectors.

Concrete use cases:

• Building a local RAG pipeline for a domain-specific knowledge base (e.g., legal documents, internal wikis) with high OOD robustness.
• Semantic search in a corpus of mixed sources where retrieval quality must degrade gracefully when queries differ from training data.
• Foundation model for embedding-based classifiers where labeled data is scarce – leverage the 4B representation.

At 4B parameters, BOOM_4B_v1 is a mid-size model that fits on consumer GPUs with reasonable quantization.

Realistic GPU: An RTX 4090 24GB can run FP16 with headroom for batching or longer contexts. An RTX 3060 12GB handles Q4_K_M well. Apple Silicon users with unified memory >16GB can run Q4_K_M comfortably.

Inference performance (estimated, Q4_K_M, batch size 1 on RTX 4090): ~200-300 tokens per second for encoding. Throughput scales with batch size – a batch of 32 documents of 512 tokens each should process at ~5000 tokens/sec.

Ollama offers the fastest path. Install Ollama, then pull the model (if available in Ollama library) or use a custom Modelfile pointing to the HuggingFace repo. Alternatively use Sentence Transformers:

1from sentence_transformers import SentenceTransformer
2model = SentenceTransformer("ICT-TIME-and-Querit/BOOM_4B_v1")
3embeddings = model.encode(["Your text here"])

For speed, enable Flash Attention 2:

1model = SentenceTransformer(
2    "ICT-TIME-and-Querit/BOOM_4B_v1",
3    model_kwargs={"attn_implementation": "flash_attention_2", "device_map": "auto"},
4    tokenizer_kwargs={"padding_side": "left"},
5)

Best quantization for most users: Q4_K_M offers the best tradeoff of quality retention (~98% of FP16) and VRAM (~3 GB). Avoid Q3_K_S for production retrieval; it may degrade OOD performance due to cumulative precision loss. For memory-constrained hardware, Q8_0 (5 GB) is safer than aggressive 4-bit.

• Size: BOOM is 2.7x larger. Expect higher representational capacity and better performance on complex retrieval tasks.
• OOD robustness: BOOM’s bagged training gives it a measurable edge in OOD benchmarks per the paper. gte-Qwen2 uses standard multi-task training.
• Speed: gte-Qwen2 is faster (lower latency, higher throughput) due to smaller size. If throughput is more critical than marginal accuracy gains, the 1.5B model may suffice.
• When to choose BOOM: When you need the best possible retrieval quality on diverse, unseen data and have the GPU memory to spare.

• Size: BOOM is nearly half the size. It runs on lower-end hardware (RTX 3060 vs. RTX 4090 for FP16).
• Performance: e5-mistral-7b often leads MTEB benchmarks due to its larger scale and instruction tuning for retrieval. BOOM’s OOD focus may close the gap on specific unseen domains.
• When to choose BOOM: If you have limited VRAM (3-5 GB) and want a high-quality embedder, BOOM is the better fit. If you have 16+ GB and need top leaderboard scores, e5-mistral-7b is the current heavyweight.

Bottom line: BOOM_4B_v1 occupies a sweet spot: near-top performance with a VRAM footprint that fits most modern consumer GPUs when quantized. Its bagging-based training is a practical alternative to increasing model size for robustness. For local RAG deployments where hardware is a constraint and OOD quality matters, this is a strong candidate.