Guidance

Key Features

What the engine gives you out of the box, in plain language.

• OpenAI-Compatible API
• NVIDIA GPU
• AMD GPU
• Apple Silicon
• CPU Inference
• Quantization
• Continuous Batching
• Multi-GPU
• Desktop GUI
• One-Line Install
• Structured Output
• Streaming

• Guided templates

  Define a structure and let the model fill only the parts you mark.

• Constrained decoding

  Restrict output to valid choices, patterns, or grammars.

• Token-efficient

  Generate only what is needed, which can lower cost and latency.

Where It Shines

The jobs this engine is best suited for.

• Reliable structured output

  Produce output that fits a template without post-processing.

• Complex prompt logic

  Combine conditionals and loops with generation in one place.

• Cost control

  Cut token use by generating only the parts that vary.

Frequently Asked Questions