End-to-End Fine-Tuning Pipeline: From Data to Deployment

Llama 3.1 8B Instruct is the sweet spot: strong instruction-following baseline, 128K context window, Apache 2.0 license (commercial-safe), and $0.10-0.20/M tokens on managed APIs vs $5-15/M for GPT-4o. LoRA on 8B typically reaches 90-95% of GPT-4o quality on narrow tasks.

Alternatives: Mistral 7B Instruct v0.3, Qwen 2.5 7B, Phi-4 14B, Llama 3.1 70B for quality-critical tasks

Unsloth provides 2x faster training and 50% less VRAM usage vs standard HF Trainer via custom CUDA kernels. On an A100 80GB, Unsloth trains 8B LoRA at ~3,000 tokens/second vs ~1,500 tokens/second vanilla. This halves your GPU costs.

Alternatives: Axolotl, LLaMA-Factory, OpenAI fine-tuning API (GPT-4o mini), Google Vertex AI fine-tuning

r=16 is the standard starting point — enough parameter capacity for task-specific adaptation without overfitting on <10K examples. QLoRA (4-bit NF4) enables 7B training on a single 24GB GPU (RTX 4090) at the cost of 5-10% quality degradation vs full-precision LoRA.

Alternatives: r=8 (smaller, faster), r=64 (more expressive), QLoRA (4-bit quantized for VRAM-constrained training)

An A100 80GB trains 8B LoRA on 5K examples in ~45 minutes (~$1.50). Full fine-tuning of 8B needs 2x A100s and takes 3-4 hours ($8-12). For 70B LoRA, use 4x A100s (~$8/hr). Lambda Labs is 30-50% cheaper than AWS for spot-equivalent on-demand GPU.

Alternatives: RunPod H100 ($3.99/hr), AWS p4d.24xlarge ($32/hr, overkill for 8B), Google Colab Pro+ (A100, $50/mo)

Generic benchmarks (MMLU, HellaSwag) don't predict task-specific fine-tune quality. Build a 200-example golden test set with expected outputs. Use Claude Haiku 4 as judge ($0.80/M tokens) to score outputs on a 1-5 rubric. LLM-as-judge correlates 0.85+ with human ratings on generation tasks.

Alternatives: ROUGE-L, BERTScore, Braintrust, Promptfoo, HELM benchmarks

Together AI serves Llama 3.1 8B at $0.20/M tokens with no infra overhead — break-even vs self-hosting at ~2M tokens/day. Above that, vLLM on Modal with A10G instances ($0.10/M tokens at 60% GPU utilization) halves serving cost. vLLM enables PagedAttention — 2-4x higher throughput vs naive serving.

Alternatives: Fireworks AI, Replicate, Hugging Face Inference Endpoints, LoRAX for multi-adapter serving

wandb captures loss curves, learning rate schedules, gradient norms, and eval metrics in real-time with minimal config (`wandb.init()` in training script). The training diff view lets you compare two runs in 30 seconds. Free tier is sufficient for most fine-tuning projects.

Alternatives: MLflow, Comet ML, TensorBoard

Mitigation: When fine-tuning erases the model's general capabilities (the model becomes narrow and brittle). Prevention: use LoRA (modifies only adapter weights), keep learning rate low (1e-4 to 3e-4), and include 5-10% general instruction-following examples in your training mix alongside task-specific data.

Mitigation: Fine-tuned model performs well on test set but fails in production because production inputs differ from training examples. Collect 10-20% of training data from actual production traffic (anonymized). Re-evaluate on production samples weekly after deployment.

Mitigation: Detectable by val loss diverging from train loss after epoch 1-2. Set early stopping with patience=3 on validation loss. For <1,000 examples, keep LoRA rank r≤8 and train for 3-5 epochs maximum. Add dropout (0.05-0.1) to LoRA adapters.

Mitigation: Your test set accidentally contains near-duplicates of training examples, inflating evals. Run MinHash deduplication across the combined train+test set before splitting. Use a 80/10/10 train/val/test split with a separate held-out production evaluation set.