LensVLM-9B

LensVLM-9B is a 9 billion parameter dense vision-language model from Apple, post-trained on top of Qwen3.5-9B-Base. It is not a general-purpose chat VLM. It is a document reader built around one specific trick: render a long document to page images at 5x, 10x, or 15x compression, let the model scan the compressed thumbnails, then use learned tools to pull only the relevant pages back to full resolution before answering.

That matters because the standard approach to long documents, tokenize everything, stuff it into a long context window, pay the prefill cost, collapses under its own arithmetic. LensVLM sidesteps it. Apple reports accuracy comparable to the full-text upper bound at roughly 4.3x effective compression, and results above retrieval-based, text-compression and visual-compression baselines up to roughly 10.1x across seven text QA benchmarks. The gains hold on multimodal documents and code understanding, and grow as compression increases.

The category it occupies: single-architecture, mid-size local VLM for multi-page document QA. Its real competitors are not chat models, they're retrieval pipelines and text-compression schemes. If your workload is "answer a question about a 200-page PDF," LensVLM is architected for exactly that, and it will beat a naive RAG stack on layouts where chunking destroys the cue. If your workload is general conversation with occasional image input, it is the wrong tool.

Weights are on Hugging Face (apple/LensVLM-9B) under the Apple Machine Learning Research Model License. The accompanying code is separate, under the Apple Sample Code License. Read both before commercial deployment, neither is a standard OSI license, and the model license inherits constraints from the Qwen base.

LensVLM-9B is dense, not Mixture-of-Experts. All 9B parameters are active on every forward pass. For local inference this has one clear consequence: VRAM scales predictably with quantisation, with no expert-routing tricks to exploit. There's no "9B total, 2B active" discount here. What you see is what you pay for.

Core specs:

• Parameters: 9B, dense
• Base model: Qwen3.5-9B-Base
• Modality: text + vision (image-text-to-text)
• Context length: not published by Apple. Plan around the visual token budget rather than a headline number, every rendered page consumes encoder tokens, and that budget is what actually limits how many pages you can scan per pass.
• Compression levels: 5x, 10x, 15x
• Tool interface: read_page, plus high-resolution image expansion
• Training cutoff: not specified

The mechanism is worth understanding before you deploy it. The vision encoder maps a fixed-size image to a fixed number of visual tokens. Rendering a page smaller therefore is compression, it is a continuous knob, not a discretised one. The failure mode is obvious: shrink characters below the encoder's effective resolution and they become unreadable. LensVLM's post-training teaches the model to tolerate that degradation on the scan pass, identify which pages look relevant from degraded thumbnails, and then call a tool to expand just those pages. Apple's analysis shows the model shifts its reliance from visual reading to expanded content as compression rises, which is the behaviour you want.

Apple also publishes tool-choice guidance that directly affects your pipeline: text expansion for rendered text, high-resolution image expansion for native documents whose layout carries task-relevant signal. Tables, forms, and multi-column PDFs are the latter case.

Practically, this means LensVLM is not a drop-in generate() call. It runs a multi-turn trajectory, scan, reason inside <think> tags, emit a tool call, receive tool output, answer. Budget for the harness, not just the weights.

The declared capability set is chat, vision, summarization, and instruction-following. In practice, the useful surface area is narrower and more valuable than that list suggests:

• Multi-page document QAthe core use case. Long reports, filings, manuals, scanned archives. Questions that require locating a fact buried on page 10 of a 15-page render.
• Summarization over long rendered textwhere you want the whole document considered, not just retrieved chunks.
• Layered-detail workflowscheap scan pass across everything, expensive resolution only where needed. This is the efficiency argument, and it's the reason to pick LensVLM over a plain 9B VLM with a big context window.
• Multimodal document understandingApple reports generalisation here, with the advantage over baselines widening at higher compression.
• Code understanding in document formrendered code pages, same scan-and-expand loop.

What it is not for: open-ended visual reasoning about photographs, real-time video, or image generation. There's no audio and no native tool ecosystem beyond the trained read/expand tools.

These are estimates derived from parameter count and standard quantisation overhead, Apple has not published an official hardware table. Add headroom for the vision encoder, KV cache, and the rendered page tokens.

Context is the wildcard. Because page images are the input, long documents inflate the KV cache fast. A 24 GB card at bf16 will fit the weights comfortably and then run out of room on a deep multi-page context.

Q4_K_M for most users. It's the standard quality-per-byte sweet spot, and for a 9B model the perplexity hit is small. Push to Q5_K_M or Q8_0 if you have the VRAM and your task is precision-sensitive, long-document fact extraction tolerates quantisation less gracefully than open chat, because the model is already reading degraded characters.

• RTX 4090 (24 GB)bf16 at full context, or Q4_K_M with room for aggressive page counts. The strongest single-GPU option here.
• RTX 4080 / 4070 Ti Super (16 GB)Q5_K_M or Q4_K_M comfortably.
• RTX 3090 (24 GB)same ceiling as the 4090, roughly 30–40% slower.
• RTX 4060 Ti 16 GB / RTX 4070 (12 GB)Q4_K_M only, shallow contexts.
• Apple M4 Max (48–128 GB unified)unified memory is a real advantage for the vision + KV footprint. bf16 is viable on 64 GB and up; throughput lags an equivalent discrete GPU.
• M4 Pro / M3 MaxQ4_K_M with modest context.

Approximate, hardware- and quantisation-dependent, generation phase only:

• RTX 4090, Q4_K_M: ~80–110 tok/s
• RTX 4090, bf16: ~40–55 tok/s
• M4 Max, bf16 via MLX: ~25–35 tok/s

Prefill is the number that will actually shape your UX. Rendered page images mean large prefill batches, and the tool-expansion round trip adds a second prefill. Measure end-to-end document latency, not tokens/s.

Apple's reference path is the official repo:

1git clone https://github.com/apple-aiml-research/ml-lensvlm
2cd ml-lensvlm
3pip install -r requirements.txt
4python scripts/run_demo.py --model apple/LensVLM-9B

For your own document:

1python demo.py --model apple/LensVLM-9B \
2    --text_file document.txt \
3    --question "What is the main finding?" \
4    --compression 10x

On Ollama: it's the lowest-friction way to run quantised 9B models locally, and once GGUF conversions of LensVLM-9B circulate it will be the fastest route to a first run. Be aware that generic llama.cpp and Ollama runtimes do not ship the trained tool-call loop, you get a VLM, not the scan-and-expand inference framework. For anything resembling production, use Apple's harness and treat Ollama as a smoke test. Start at --compression 10x; drop to 5x if accuracy is marginal, climb to 15x only when context pressure forces it.

vs. Qwen3.5-9B-Basethe same backbone without the vision post-training. Identical parameter count, so identical VRAM and roughly identical throughput at equivalent quantisation. Qwen3.5-9B is text-in, text-out: faster prefill, no encoder overhead, and the right choice when your documents are already clean digital text and your context window can hold them. LensVLM wins when documents are long, page-structured, or image-native, or when tokenizing the full corpus is the bottleneck.

vs. Gemma 3 12Ba larger general-purpose VLM with a broader multimodal surface and a permissive license. It handles photographs, charts, and general visual chat that LensVLM was never trained for. It does not implement selective expansion, so long-document QA means either a very large context or an external retrieval layer, and Apple's results put LensVLM ahead of retrieval baselines as compression climbs. Choose Gemma 3 12B for breadth and licensing simplicity; choose LensVLM-9B for depth on long page-structured documents.

The honest framing: LensVLM-9B is a specialist with a narrow but deep advantage. If your problem is multi-page document QA and you're willing to run the harness, it's the better architecture. If you need one model to do everything, it isn't.