Single 7B model unifying generation and embedding via Generative Representational Instruction Tuning.
Access model weights, configuration files, and documentation.
See which devices can run this model and at what quality level.
GritLM-7B unifies text generation and representation in one Mistral-7B-based model via Generative Representational Instruction Tuning (GRIT), using bidirectional attention for embeddings and causal attention for generation, with task selected by the instruction. Set a new MTEB SOTA at release while matching instruction-tuned generative performance, and notably accelerates RAG by >60% on long documents because the same model serves both retrieval and generation.
View on Hugging Face