Fine-Tuning LLMs

About This Course

Fine-tuning is how you make a general-purpose LLM a specialist. This course covers the complete fine-tuning workflow: dataset curation and formatting, training with LoRA and QLoRA for efficiency, evaluating trained models, and deploying them to production. You will work with Llama 3, Mistral 7B, and the OpenAI fine-tuning API on real tasks like domain-specific question answering, style transfer, and structured output consistency. New to these terms? LoRA (Low-Rank Adaptation) is a technique that fine-tunes a model by training small adapter matrices instead of all the weights — making fine-tuning feasible on a single GPU. QLoRA adds 4-bit quantization on top of LoRA, reducing memory further. PEFT (Parameter-Efficient Fine-Tuning) is the family name for these techniques. TRL is Hugging Face's training library for supervised fine-tuning and RLHF. vLLM is the inference engine used to serve your fine-tuned model at production throughput. All are explained from first principles inside the course, but knowing these names upfront will make the curriculum map easier to follow.

What You'll Learn

• Decide when fine-tuning is worth it vs. prompting or RAG
• Curate and format high-quality training datasets for any task
• Fine-tune Llama 3 and Mistral with LoRA for efficient training
• Apply QLoRA to fine-tune on consumer hardware (single A100)
• Use the Hugging Face ecosystem fluently: Transformers, PEFT, TRL
• Track experiments with Weights & Biases
• Evaluate fine-tuned models rigorously with held-out test sets
• Deploy fine-tuned models with vLLM for production serving

Who Is This For?

Prerequisites

• Understanding LLMs
• Python for AI
• PyTorch fundamentals required — this is a hands-on ML engineering course, not an intro course. If you haven't trained a model in PyTorch before, complete a PyTorch basics tutorial first
• GPU compute access recommended (Google Colab Pro or equivalent) — QLoRA fine-tuning requires significant compute

Tools & Technologies