Axolotl

Axolotl is an open-source fine-tuning framework maintained by Axolotl AI, not an inference engine. This distinction matters. Unlike vLLM or Ollama, which serve trained models, Axolotl is the tool you use before inference: it prepares your custom model by fine-tuning an open-weight base on your data. The result is a model you then serve with a dedicated inference engine.

First released in 2023 and licensed under Apache 2.0, Axolotl has grown to over 12,000 GitHub stars with 246 contributors and nearly 20,000 monthly PyPI downloads. Written in Python, it abstracts away the boilerplate of distributed training, letting you define the entire fine-tuning pipeline in a single YAML config file. The team behind it is focused on making fine-tuning accessible to engineers who need custom models without building training infrastructure from scratch.

Axolotl occupies the fine-tuning category. It competes with frameworks like Hugging Face Transformers + Accelerate, Unsloth, and TRL. Its design philosophy centers on configurability and reproducibility: you version a YAML file, not a training script.

You interact with Axolotl through a YAML configuration file that describes three things: the base model, your training data, and the fine-tuning method. A typical config specifies the Hugging Face model ID or local path, the dataset location and formatting, the PEFT method (LoRA, QLoRA, or full fine-tuning), and training hyperparameters like learning rate, batch size, and number of epochs.

The runtime model is straightforward. You install Axolotl with pip install axolotl, write your config, and run accelerate launch -m axolotl.cli.train your_config.yml. Axolotl handles loading the model, tokenizing the data, setting up the optimizer and scheduler, and executing the training loop. For multi-GPU setups, it integrates with FSDP and DeepSpeed under the hood.

Axolotl does not expose an API or serve models. After training completes, you save the adapter weights (for LoRA) or the full model checkpoint and load them into an inference engine. There is no built-in OpenAI-compatible server, no continuous batching, and no inference-time functionality. It is a training tool, period.

Axolotl is built for NVIDIA GPUs. The confirmed capabilities include NVIDIA GPU support, quantization, and multi-GPU training. There is no support for CPU, Apple Silicon, or AMD GPUs in the official release, though community efforts exist for ROCm compatibility on AMD hardware such as the Ryzen AI Max+ 395.

For memory efficiency, Axolotl supports QLoRA, which quantizes the base model to 4-bit or 8-bit precision using bitsandbytes, dramatically reducing VRAM requirements. This lets you fine-tune 7B parameter models on a single consumer GPU with 24 GB VRAM. Larger models or full fine-tuning require multi-GPU setups.

Multi-GPU scaling uses FSDP (Fully Sharded Data Parallelism) and DeepSpeed ZeRO stages. Axolotl handles the coordination, but you need to configure the parallelism strategy in your YAML or a separate DeepSpeed config file. The framework does not provide its own continuous batching or inference-time optimization; those are the domain of the serving engine you use afterward.

Training speed depends on your hardware, model size, and method. QLoRA on a single A100 80 GB can fine-tune a 70B model, while full fine-tuning of the same model typically requires multiple A100s or H100s with FSDP. Axolotl does not publish throughput benchmarks, but community reports indicate it is competitive with other PyTorch-based fine-tuning frameworks.

Config-driven training. The entire training pipeline, from model selection to data preprocessing to optimization method, lives in one YAML file. This makes runs reproducible and easy to share across a team.

Multiple fine-tuning methods. Axolotl supports LoRA, QLoRA, and full fine-tuning out of the box. Switching between methods requires editing a few lines in the config, not rewriting training code.

Multi-GPU training. FSDP and DeepSpeed integration let you scale from a single GPU to multiple nodes. Axolotl abstracts the distributed setup so you declare the strategy in config rather than writing distributed launch scripts.

Quantization support. QLoRA with bitsandbytes reduces VRAM usage significantly, enabling fine-tuning on smaller GPUs. The framework also supports quantization-aware training for post-training compression.

Broad model compatibility. Axolotl supports most open models available through Hugging Face Transformers, including Llama, Mistral, Qwen, Gemma, Phi, Falcon, and others. Community contributions extend support to newer architectures regularly.

Custom model training on domain data. Teams fine-tune open models on proprietary datasets for tasks like code completion, legal document analysis, or customer support. The YAML config makes it easy to iterate on data and hyperparameters.

Reproducible fine-tuning pipelines. Engineering teams version their YAML configs alongside training data. This enables repeatable runs for experimentation, regression testing, and production model retraining.

Exploration of fine-tuning methods. Researchers and practitioners use Axolotl to compare LoRA, QLoRA, and full fine-tuning on the same base model and data, controlled by config changes alone.

Resource-constrained fine-tuning. With QLoRA, teams fine-tune 7B to 13B models on single GPUs with 24 GB VRAM, avoiding the need for multi-GPU infrastructure.

Axolotl is a poor fit for inference. Do not use it for serving, batch prediction, or real-time model access. You need a separate inference engine to run the resulting model.

The install command is pip install axolotl. For GPU support, ensure you have CUDA, PyTorch with CUDA, and bitsandbytes installed. A typical workflow:

1. Install Axolotl: pip install axolotl
2. Write a YAML config file specifying your base model, dataset, and training method.
3. Run training: accelerate launch -m axolotl.cli.train config.yml
4. Save the trained adapter or model checkpoint.
5. Load the checkpoint into an inference engine for serving.

The smallest meaningful path requires a GPU, a Hugging Face model ID or local model file, a formatted dataset, and the Axolotl config. The official documentation at docs.axolotl.ai covers config options and examples. Community support is active on the Axolotl Discord.

Axolotl vs Unsloth. Both target configurable fine-tuning with a focus on speed and memory efficiency. Unsloth offers optimized kernels for LoRA and QLoRA that can be faster than Axolotl on supported operations, and it supports more hardware including Apple Silicon. Axolotl offers broader model support and deeper integration with FSDP and DeepSpeed for multi-GPU scaling. Choose Axolotl if you need to fine-tune on a wider range of models or scale to multiple GPUs. Choose Unsloth if you optimize for single-GPU speed or need Apple Silicon support.

Axolotl vs Hugging Face Transformers + Accelerate. Transformers provides the building blocks; you write training scripts from scratch. Axolotl wraps those building blocks into a config-driven interface. Choose Axolotl if you want to avoid writing training boilerplate and value reproducibility through config files. Choose raw Transformers if you need full control over every training detail or are integrating with custom training logic.

Axolotl vs vLLM or Ollama. These are not alternatives. vLLM and Ollama serve models. Axolotl trains them. You use Axolotl first to create a custom model, then serve it with vLLM or Ollama. Do not evaluate them as competing choices for the same task.