Solo Leveling GPT-2 Chatbot

Overview

Solo assignment for the ML2 course at INN. Starting from a base GPT-2 project provided by the course teacher, I created a custom dataset of 44 Solo Leveling Q&A pairs in CSV format, built a CSV loader to replace the hardcoded training data, and fine-tuned the pretrained GPT-2 model using loss masking so only the answer tokens contribute to training loss. After fine-tuning for 3 epochs, the model can answer questions about Solo Leveling characters, abilities, and lore through an interactive terminal chatbot.

My Contributions

• Created a custom 44-pair Q&A dataset about the Solo Leveling universe in CSV format
• Built a load_csv() function to replace hardcoded training data with dynamic CSV loading
• Configured fine-tuning with loss masking — prompt tokens masked with -100 so only answer tokens contribute to loss
• Fine-tuned GPT-2 for 3 epochs using AdamW optimizer with gradient clipping
• Built and tested an interactive terminal chatbot that responds to Solo Leveling lore questions

Tech Breakdown

Python, PyTorch, Hugging Face Transformers (GPT2LMHeadModel, GPT2Tokenizer), fine-tuning with loss masking, AdamW optimizer, top-p sampling (nucleus sampling) for generation, custom Dataset class with padding and attention masks