MLX

Key Features

What the engine gives you out of the box, in plain language.

• OpenAI-Compatible API
• NVIDIA GPU
• AMD GPU
• Apple Silicon
• CPU Inference
• Quantization
• Continuous Batching
• Multi-GPU
• Desktop GUI
• One-Line Install
• Structured Output
• Streaming

• Unified memory

  Shares memory between CPU and GPU on Apple Silicon, avoiding slow copies.

• mlx-lm runner

  Run and serve language models with a few lines of Python or a local server.

• On-device fine-tuning

  Train and adapt models locally on a Mac, no cloud GPU required.

Where It Shines

The jobs this engine is best suited for.

• Running models on a Mac

  Get the most performance from an M-series Mac for local model work.

• On-device experiments

  Prototype with private data on a laptop with no cloud bill.

• Local fine-tuning

  Adapt a small model to your data on Apple hardware.

Frequently Asked Questions