Key: Implement attention from nn.Linear + matrix multiply + causal mask.
When you finally find that elusive , you will notice what is missing . Do not be alarmed. This is a feature, not a bug. Build A Large Language Model -from Scratch- Pdf -2021