Guidance and LMQL compared side by side on GitHub stars, downloads, language, license, capabilities, strengths and trade-offs.
TL;DR
Choose Guidance if you want an actively maintained Python library to interleave generation, regex constraints, and application logic directly inside your codebase. Choose LMQL if your team prefers a declarative, SQL-style query language with expressive constraint clauses and advanced decoders like beam search. For production reliability and broader community support, Guidance is the safer default.
Add an Engine
Slot 3 of 3
Comparing inference engines is the process of evaluating two or three competing tools side by side on live community signals, technical capabilities, language, license, and trade-offs so a team can pick the one that fits its hardware and workload.
Start with the constraints that are not negotiable: the hardware you already run on, the license your legal team will sign off on, and the kind of workload you need to serve. An engine that cannot use your GPU or fit your model is not really an option, no matter how popular it is.
Then look at live signals: GitHub stars and contributors show whether the project is gathering momentum, PyPI downloads show whether teams are actually shipping with it, and last-commit date shows whether the maintainers are still around. Capability flags like OpenAI-compatible API, GPU support, quantization, continuous batching, and multi-GPU narrow the field to engines that match your job-to-be-done.
Finish by reading the strengths and trade-offs columns side by side. The simplest engine that covers your real requirements almost always beats the most powerful one. Copy the share link once you have a comparison you trust so you can revisit it during planning.
Decisions about inference engines rarely happen in isolation. Pair this comparator with the directories and benchmarks that ground the rest of the stack.
Live data from GitHub and the package registries, refreshed every day.
GitHub stars
Contributors
Forks
Star history sampled from the GitHub stargazer timeline and extended by our daily sync.
Guidance and LMQL take two distinct approaches to the problem of structured generation. Guidance treats constrained generation as an embedded Python control flow where prompt text, regular expressions, and host code interleave in a continuous stream. LMQL treats prompting as a declarative programming language where you write prompt queries with explicit constraint clauses and custom decoding strategies. Both tools eliminate unstructured text parsing by forcing models to adhere to defined rules during decoding.
The central architectural difference lies in how each tool expresses program logic and constraints. Guidance is a Python library maintained by Guidance AI. You write standard Python code that chains strings, variables, and generation directives such as gen() and select(). Because it operates directly inside Python, control flow relies on standard language constructs like loops, conditionals, and context managers. Guidance also features token healing, a technique that rolls back prompt-boundary tokenization to prevent biased completions caused by artificial token boundaries.
LMQL, developed by the SRI Lab at ETH Zurich, implements a standalone domain-specific language inspired by SQL and Python. In LMQL, prompts are structured queries containing bracketed variables (such as
Open and closed model rankings with benchmarks, context windows, modalities, and live API prices.
Straight answers to the questions we hear most often.
Can't find what you're looking for? Book a Discovery Call
Guidance is generally the better choice for production systems due to its larger community, active maintenance, and tighter integration with standard Python code. LMQL offers a cleaner declarative syntax for multi-variable prompts with where clauses, but its development activity has slowed.
No, you would not use them together in the same generation step. Both libraries provide distinct runtimes and syntaxes for controlling output tokens. You should select one approach based on whether you prefer embedded Python templates or declarative query syntax.
Neither framework publishes formal head-to-head throughput benchmarks, and raw inference speed is primarily determined by your underlying model backend (such as llama.cpp or Hugging Face Transformers). However, Guidance avoids parsing a separate query language, which can reduce CPU-side framework overhead during execution.
Both frameworks save tokens by generating only necessary values and skipping static text. However, when using hosted commercial APIs like OpenAI, multi-step programmatic control flows in either tool can require multiple sequential API round trips, increasing overall cost compared to single-shot API calls.
Guidance is easier to learn for Python engineers because it operates as a standard Python library using standard control flow. LMQL requires learning a specialized domain-specific query language that resembles a mix of Python and SQL.
Switching requires rewriting your prompt templates and constraint definitions. Guidance relies on embedded Python primitives like gen() and select(), whereas LMQL uses top-level query strings with where clauses. Because the underlying model backends are similar, your infrastructure setup can remain largely intact.
Guidance is more popular on GitHub with 21.8K stars, against 4.2K for LMQL. Stars measure developer interest, so also compare downloads and contributors above.
Guidance has 84 contributors and its last commit was 6 mo ago. LMQL has 41 contributors and its last commit was 1 yr ago.
LMQL was first released in 2022 and Guidance in 2023. A longer track record usually means more examples, integrations and answered questions.
Guidance is a Python engine from Guidance AI, released under the MIT license. LMQL is a Python engine from ETH Zurich (SRI Lab), released under the Apache 2.0 license. The table above shows how they differ on hardware support, APIs and serving features.
Start with your hardware and your traffic. Check which engine supports your GPUs, whether you need an OpenAI-compatible API, and how many users it must serve at once. Then compare community activity, since an active project ships fixes faster.
PyPI downloads per month
[RESPONSE]whereAdoption and ecosystem activity differ substantially. At the time of writing, Guidance has accumulated 21,786 GitHub stars, 84 contributors, and 11,757 monthly PyPI downloads since its initial release on May 16, 2023. LMQL, first released on December 1, 2022, holds 4,218 GitHub stars, 41 contributors, and 850 monthly PyPI downloads. Guidance has maintained a faster development pace, while LMQL development has slowed.
| Dimension | Guidance | LMQL |
|---|---|---|
| Primary Architecture | Python library and DSL | Declarative prompt query language |
| Constraint Mechanism | Regex, CFGs, select options, token healing | Logical where constraints, token masks |
| Execution Style | Procedural Python execution | Query compilation with constraint evaluation |
| Decoding Algorithms | Greedy, sampling, backend defaults | Argmax, sample, beam search, best_k |
| GitHub Stars (measured at writing) | 21,786 | 4,218 |
| PyPI Downloads / Month (measured at writing) | 11,757 | 850 |
| License | MIT | Apache-2.0 |
Neither Guidance nor LMQL functions as an independent high-throughput inference server with continuous batching or paged attention. Both systems operate as client-side or engine-side wrappers that steer local model backends or cloud endpoints.
No standardized, independent head-to-head throughput benchmarks exist comparing Guidance and LMQL on identical hardware. Performance in both frameworks depends on how efficiently their masking engines translate constraints into token logit processors without stalling the forward pass. Guidance minimizes overhead by enforcing regular expressions and context-free grammars directly on local model tokenizers, while skipping the generation of static structural tokens entirely. LMQL accelerates execution through an asynchronous runtime that can evaluate multi-variable queries and run multiple search branches concurrently. When running against remote APIs like OpenAI, both frameworks face latency overhead from multi-turn network requests whenever programmatic control flow requires intermediate model responses.
Guidance supports local model execution via Hugging Face Transformers, llama.cpp, and generic CPU inference, alongside remote backends such as OpenAI and Azure OpenAI. When running locally, Guidance can execute on Nvidia GPUs and standard x86 CPUs. Because Guidance acts on token probabilities directly, its most capable grammar and regex features function best when using local weights or backends that expose logit biases.
LMQL similarly interfaces with Hugging Face Transformers, llama.cpp, and OpenAI endpoints. When running local models, LMQL supports Nvidia GPU acceleration and CPU execution. Its constraint engine evaluates character-level and token-level conditions, translating them into dynamic token masks. However, both frameworks face functional limits when targeting hosted black-box APIs: proprietary commercial APIs do not permit custom grammar-based logit masking, limiting both engines to prompt-level steering, schema enforcement features provided by the vendor, or sequential API calls.
Neither tool is an OpenAI-compatible HTTP inference server. They do not replace serving frameworks like vLLM or TGI. Instead, you deploy both tools inside your application layer as Python libraries.
Guidance integrates into existing backend services as ordinary application code. You instantiate a model object, compose your guided prompt template, and execute it synchronously or via streaming within your FastAPI, Flask, or worker processes. LMQL can be executed via its Python integration using the @lmql.query decorator, or run through its command-line interface and standalone web-based Playground IDE. LMQL queries return structured result objects containing assigned variables, logprobs, and execution traces.
Both libraries install through standard Python package management via pip install guidance and pip install lmql. Setting up basic scripts against OpenAI or local models requires only a few lines of code in either case.
Guidance requires less mental overhead for Python developers. Because you write standard Python with overloaded operators and helper functions, debugging follows standard Python stack traces. The primary learning curve involves understanding how Guidance interleaves static text chunks with generation directives.
LMQL introduces a higher barrier to entry because developers must learn its query syntax, variable binding semantics, and constraint rules. While LMQL provides an interactive web playground to visualize decoding and constraint verification, integrating multi-step LMQL queries into complex codebases requires managing its custom runtime semantics and query return objects.
Guidance is licensed under the permissive MIT license. This allows unrestricted commercial use, modification, and distribution in proprietary software systems.
LMQL is distributed under the Apache-2.0 license. Apache-2.0 is also a permissive open-source license that allows commercial use and modification, while including explicit terms covering patent grants and trademark restrictions. Both licenses are fully suitable for enterprise commercial adoption.
Choose Guidance if:
Choose LMQL if:
where clause rather than building complex regular expressions.