TRL

Key Features

What the engine gives you out of the box, in plain language.

• OpenAI-Compatible API
• NVIDIA GPU
• AMD GPU
• Apple Silicon
• CPU Inference
• Quantization
• Continuous Batching
• Multi-GPU
• Desktop GUI
• One-Line Install
• Structured Output
• Streaming

• Supervised fine-tuning

  A trainer for standard fine-tuning on your own data.

• Preference alignment

  Methods like DPO and GRPO align a model to preferred outputs.

• Transformers integration

  Works directly with Hugging Face models and libraries.

Where It Shines

The jobs this engine is best suited for.

• Aligning a model

  Tune a model toward preferred answers with DPO or similar.

• Custom fine-tuning code

  Build a training pipeline on solid, tested components.

• Research and experiments

  Try new alignment methods with ready-made trainers.

Frequently Asked Questions