promptdojo_

Attention and Transformer blocks — step 2 of 7

ch 44 · cnns, transformers, and useful llm internals2/7

Doubling a transformer's context window from 100k to 200k tokens does what to attention's compute cost, and why?

read, then continue.