SFT with LoRA, DPO and GRPO on a pretrained model — rewards, preferences and the hard question: better at the task, or just better at the reward?
What we covered
How do you train a model to get better at a task? We worked through it together, starting from a pretrained model and training it ourselves.
- Supervised Fine-Tuning with LoRA — training on examples, and what changes inside the model when you do.
- DPO for learning from preferences — which answers deserve a reward, and how to teach that choice.
- GRPO for reinforcement learning with rewards — where things get tricky, and what you need to try this yourself.
- Reading the numbers — how do you know the model actually improved? Comparing results, making sense of the metrics.
- Reward vs reality — the easy-to-miss question when training looks good: is the model better at the task, or just better at earning the reward?
What we built
A hands-on training arc in code — SFT with LoRA on examples, preference learning with DPO, reward-driven updates with GRPO — with results compared live in the room. Certificates for attendees who completed the session and the closing form.
Takeaways
- Training after pre-training is a sequence of choices: examples, preferences, rewards — each with its own failure modes.
- The numbers only mean something if you interrogate them — reward going up is not the same as getting better.
- 286 learners, one room, code on every screen — the AIYatra format at full strength.
Join the next one
We run sessions like this every Saturday in Hyderabad — free, hands-on, and open to everyone. RSVP on Meetup and bring a laptop with Python 3.10+.
