The world around us
20172017: The Transformer architecture
Attention became the basis of a new approach to sequence processing.
The 2017 paper Attention Is All You Need introduced the Transformer architecture and tested it on machine translation.
Attention captures relationships between parts of an input sequence. This organisation supports parallel computation and became an important direction for neural-network development.
Sources
AI-generated photorealistic collage illustrating the theme of the year.