Why Axolotl
Axolotl abstracts the complexity of fine-tuningFine-tuningThe process of further training a pre-trained model on a smaller, task-specific dataset to specialize its behavior for a particular use case.Learn more → setup into a single YAML config file. Instead of writing Python training scripts, you describe your model, dataset, LoRALoRALow-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that trains small weight matrices while freezing the base model.Learn more → settings, and training parametersParametersThe numerical weights inside a neural network that are learned during training — the 'knowledge' of the model, measured in billions for modern LLMs.Learn more → in YAML. Axolotl handles dataset formatting, model loading, distributed training setup, and checkpointCheckpointA saved snapshot of model weights and training state that allows resuming training, evaluation, or deployment from a specific point.Learn more → saving.
Axolotl supports every major fine-tuning technique: LoRA, QLoRA, full fine-tuning, and DPO — all switchable by changing a few config lines. It also integrates with Unsloth for 2–5× faster training on supported models.
Install Axolotl
Clone and install: `git clone https://github.com/axolotl-ai-cloud/axolotl && cd axolotl && pip install -e '.[flash-attn,deepspeed]'`. Flash Attention 2 is strongly recommended — it speeds up training by 30–50% and reduces memory usage.
Verify GPU setup: `python -c 'import torch; print(torch.cuda.is_available(), torch.cuda.device_count())'`. Axolotl works on 1 to 8+ GPUs. For multi-GPU, install DeepSpeed: `pip install deepspeed`.
Write a config file
Create a YAML config file. Minimal QLoRA example: `base_model: mistralai/Mistral-7B-Instruct-v0.3; load_in_4bit: true; adapter: qlora; lora_r: 16; lora_alpha: 32; datasets: [{path: my_dataset, type: alpaca}]; num_epochs: 3; micro_batch_size: 2; gradient_accumulation_steps: 4; output_dir: ./output`.
The `type` field in `datasets` specifies the formatting. Common values: `alpaca` (instruction/input/output), `sharegpt` (chat format with turns), `completion` (raw text). Axolotl automatically applies the correct chat template for each model family.
Run training
Single GPU: `axolotl train config.yaml`. Multi-GPU with DeepSpeed: `axolotl train config.yaml --deepspeed deepspeed_configs/zero2.json`. Axolotl saves checkpoints to `output_dir` and logs to Weights & Biases if configured.
For large models or long training runs, enable gradient checkpointing (`gradient_checkpointing: true`) to trade compute for memory. This allows training models that would otherwise OOM, at the cost of ~30% slower training speed.
Export and deploy
After training, merge the LoRA weights into the base model: `axolotl merge-lora config.yaml --lora-model-dir ./output`. This creates a full model that can be deployed without the adapter overhead.
Export to GGUFGGUFA file format for distributing quantized language models optimized for efficient local inference, popularized by llama.cpp.Learn more → for local deployment: `python convert_hf_to_gguf.py ./merged-model --outtype q4_k_m --outfile ./my-model-q4.gguf`. Then load it with Ollama: `ollama create my-model -f ./Modelfile` where the Modelfile points to your GGUF file.