Programming Tutorials

  You have curated a high-quality instruction dataset. You have set up your QLoRA config. You launch   SFTTrainer , and within seconds, your training loop crashes with an   IndexError: index out of range , or worse, your loss flatlines at   0.0   or   NaN . This is the most common bottleneck engineers face when migrating from Llama 2 to Llama 3 or 3.1. The issue isn't your dataset quality; it is a fundamental misalignment between the Llama 3.1 tokenizer’s special tokens, the default padding behavior in Hugging Face’s  transformers  library, and how the model interprets "End of Turn" versus "End of Text." This guide details the root cause of these convergence failures and provides the production-grade code required to fix them. The Root Cause: Why Llama 3.1 Breaks Standard Pipelines The Llama 3 family introduced a massive vocabulary expansion (128k tokens) and a shift in special token usage. In older models (and Llama 2), the End of Sentence (EOS) ...