Английская Википедия:Attention Is All You Need

Содержание

Текст

Текст

Текст

Текст

Текст

Текст

Текст

Текст

Текст

Файл:The-Transformer-model-architecture.png

An illustration of main components of the transformer model from the paper

"Attention Is All You Need" is a 2017 research paper by Google. Authored by eight scientists, it was responsible for expanding 2014 attention mechanisms proposed by Bahdanau et. al. into a new deep learning architecture known as the transformer. The paper is considered the founding document for modern artificial intelligence, as transformers became the main architecture of large language models. At the time, the focus of the research was on improving Seq2seq techniques for machine translation, but even in their paper the authors saw the potential for other tasks like question answering and for what is now called multimodal Generative AI.

Шаблон:As of the paper was cited more than 80,000 times. All eight authors left Google.

References

Шаблон:Ai-stub