Artificial Intelligence
Advanced
Fine-Tuning Open Source LLMs
Master parameter-efficient fine-tuning (PEFT) and LoRA techniques on open-weights models.
Course Overview
Prepare instruction datasets, optimize memory efficiency with QLoRA, and fine-tune open language models like Llama and Mistral for specific enterprise domain tasks.
What You Will Learn
Prepare, clean, and format instruction datasets for SFT (Supervised Fine-Tuning)
Apply Low-Rank Adaptation (LoRA) and 4-bit quantization (QLoRA)
Fine-tune open-weights models like Llama 3 and Mistral using HuggingFace TRL
Evaluate fine-tuned models using perplexity and task-specific benchmarks
Export merged LoRA weights for efficient production deployment
Tools & Technologies Used
PyTorch
HuggingFace Transformers
TRL
PEFT
BitsAndBytes
Weights & Biases
Structured Curriculum
2 Modules · 4 Lessons · 270 Minutes Total
Module 1
Module 1: Dataset Curation & LoRA Architecture
2 lessons
Module 1: Dataset Curation & LoRA Architecture
Format instruction datasets and configure parameter-efficient adapters.
-
Instruction Dataset Preprocessing & Tokenization
Format multi-turn conversation datasets into standardized ChatML structures.
-
Understanding LoRA & QLoRA Quantization
Deconstruct adapter rank, alpha parameters, and 4-bit NormalFloat quantization mechanics.
Module 2
Module 2: SFT Training & Model Evaluation
2 lessons
Module 2: SFT Training & Model Evaluation
Execute fine-tuning runs, track metrics, and export final model weights.
-
Supervised Fine-Tuning with HuggingFace TRL
Train a LoRA adapter on single-GPU hardware using SFTTrainer.
-
Evaluation, Weight Merging & Inference
Evaluate fine-tuned model loss, merge adapter weights into base models, and run vLLM inference.
Practical Project & Capstone Outcome
🚀 Capstone Project
Domain-Adapted Financial Q&A Language Model
Curate a specialized financial dataset, fine-tune a Llama-3 model using QLoRA, evaluate output precision against the base model, and deploy it for local inference.
Prerequisites
- Python proficiency
- Basic knowledge of PyTorch and deep learning concepts
Intended Audience
- Machine Learning Engineers customizing open-source models for domain tasks
- Data Scientists seeking cost-effective alternatives to commercial LLM APIs
Instructor Information
D
Dr. Aris Thorne
Course Author & Industry Expert
Dr. Aris Thorne is an LLM Research Engineer specializing in parameter-efficient fine-tuning and model alignment techniques.
Frequently Asked Questions
Can I complete this course using free Google Colab T4 GPUs?
Yes! All exercises are optimized to run within free T4 GPU memory limits using 4-bit QLoRA.