Difference between revisions of "Eosc/pretraining/nvidia"
(Created page with "= Background = This page provides a recipe to large-scale pre-training of a BERT neural language model, using the [https://github.com/NVIDIA/DeepLearningExamples/tree/master/...") |
(→Background) |
||
| Line 2: | Line 2: | ||
This page provides a recipe to large-scale pre-training of a | This page provides a recipe to large-scale pre-training of a | ||
| − | BERT neural language model, using the | + | BERT neural language model, using the high-efficiency |
[https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT NVIDIA BERT] implementation | [https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT NVIDIA BERT] implementation | ||
(which is based on TensorFlow, in contrast to the | (which is based on TensorFlow, in contrast to the | ||
Revision as of 10:11, 7 November 2020
Background
This page provides a recipe to large-scale pre-training of a BERT neural language model, using the high-efficiency NVIDIA BERT implementation (which is based on TensorFlow, in contrast to the NVIDIA Megatron code).
