Английская Википедия:DeepSpeed
Шаблон:Short description Шаблон:Infobox software DeepSpeed is an open source deep learning optimization library for PyTorch.[1] The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware.[2][3] DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 1 trillion or more parameters.[4] Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under MIT License and available on GitHub.[5]
The team claimed to achieve up to a 6.2x throughput improvement, 2.8x faster convergence, and 4.6x less communication.[6]
See also
References
Further reading
External links
- AI at Scale - Microsoft Research
- GitHub - microsoft/DeepSpeed
- ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters - Microsoft Research
Шаблон:Deep Learning Software Шаблон:Microsoft FOSS Шаблон:Microsoft Research
Шаблон:Microsoft-software-stub
- Английская Википедия
- C++ libraries
- Python (programming language) scientific libraries
- Free and open-source software
- Microsoft development tools
- Microsoft free software
- Microsoft Research
- Software using the MIT license
- 2020 software
- Deep learning software
- Software using the Apache license
- Страницы, где используется шаблон "Навигационная таблица/Телепорт"
- Страницы с телепортом
- Википедия
- Статья из Википедии
- Статья из Английской Википедии
