From “Attention Is All You Need” to running source code — the transformer, line by line, from paper to PyTorch.
What we covered
The paper that started it all, read the way it deserves: with an editor open beside it. We walked the original transformer section by section and implemented as we went.
- Attention, from the equation outward — scaled dot-product attention derived, then coded, then tested against PyTorch’s own.
- Multi-head as parallel subspaces — why heads split the work and how they rejoin.
- Positionals, residuals, norms — the unglamorous pieces that make depth trainable.
- The full encoder block — assembled live, forward pass verified end to end.
What we built
A working transformer encoder block, built line-by-line from the paper — 56 learners watching each equation become running code on their own machines.
Takeaways
- Papers are specifications, not scripture — implement as you read and confusion becomes debugging.
- The transformer is six ideas stacked well. None of them requires permission to understand.
- This session set the template for everything AIYatra does: paper → code → running laptop.
Join the next one
We run sessions like this every Saturday in Hyderabad — free, hands-on, and open to everyone. RSVP on Meetup and bring a laptop with Python 3.10+.
