Attention Is All You Need
"dispensing with recurrence and convolutions entirely"
This was the bold bet that paid off. In 2017, dropping LSTMs felt risky — every major NLP lab was invested in recurrence. The fact that they went all-in on attention is what made this paper a paradigm shift, not just an improvement.
▲ 87discuss →