In the context of the EOSC-Nordic EasyBuild pilot, the following page provides instructions for how to benchmark different software configurations on typical problems that are likely to affect NLPL users. Relevant variation will typically contrast pre-compiled binary installations (e.g. `pip` wheels) vs. locally compiled modules, where architecture-specific optimizations are enabled and optimized libraries (e.g. Intel MKL) are used.
We use this Python script which tests matrix multiplications and singular value decomposition (SVD).
OpenBLAS on Saga
IMKL on Saga
We use training an ELMo model on 2 GPUs as a benchmark. Any plain text corpus can be fed to this code.