LocalAI

Key Features

What the engine gives you out of the box, in plain language.

• OpenAI-Compatible API
• NVIDIA GPU
• AMD GPU
• Apple Silicon
• CPU Inference
• Quantization
• Continuous Batching
• Multi-GPU
• Desktop GUI
• One-Line Install
• Structured Output
• Streaming

• OpenAI-compatible API

  Mirrors the OpenAI API so apps switch over with no code changes.

• Runs anywhere

  Works on CPU and on NVIDIA, AMD, and Apple Silicon, no GPU required.

• Many model types

  Serve text, image, and audio models from one endpoint.

Where It Shines

The jobs this engine is best suited for.

• Private OpenAI replacement

  Keep data in house by pointing apps at your own endpoint.

• CPU-only deployments

  Run models on servers with no GPU at all.

• All-in-one local AI

  Serve text, image, and audio models from a single tool.

Frequently Asked Questions