Infrastructure/software/glibc

From Nordic Language Processing Laboratory
(Difference between revisions)
Jump to: navigation, search
(Installation on Abel)
(Installation on Taito)
Line 78: Line 78:
 
<pre>
 
<pre>
 
cp -pr /etc/ld.so.conf* /proj/nlpl/software/glibc/2.18/etc
 
cp -pr /etc/ld.so.conf* /proj/nlpl/software/glibc/2.18/etc
 +
echo "/lib64" >> /proj/nlpl/software/glibc/2.18/etc/ld.so.conf
 
/proj/nlpl/software/glibc/2.18/sbin/ldconfig  
 
/proj/nlpl/software/glibc/2.18/sbin/ldconfig  
 
</pre>
 
</pre>

Revision as of 16:16, 26 September 2018

Contents

Background

The operating system on Abel dates back to the original year of installation (2012, for all we recall). While many development tools and libraries are available in newer versions (through the module system), the version of the basic GNU C Library is intricately linked to the Linux kernel. Some packages that are distributed in pre-compiled form (typically as Python wheels) require more recent versions of the C Library. A little bit of trickery makes it possible to get these binaries to execute on Abel, using a custom, NLPL-specific installation of the GNU C Library and its dynamic linker.

Usage on Abel

To make a binary use our custom installation of the GNU C Library, we need to make sure the modern version of the dynamic linker is used, with the location of the modern C Library at the front of the dynamic library load path. For example:

#!/bin/sh

exec /projects/nlpl/software/glibc/2.18/lib/ld-linux-x86-64.so.2 \
  --library-path /projects/nlpl/software/glibc/2.18/lib:${LD_LIBRARY_PATH} \
  /projects/nlpl/software/tensorflow/1.11/bin/.python3.5 "$@"

We should generalize the above a little more, i.e. make the script determine the actual binary to be invoked as a function of $0.

Installation on Abel

wget https://ftp.gnu.org/gnu/glibc/glibc-2.18.tar.bz2
tar jpSxf glibc-2.18.tar.bz2
cd glibc-2.18
mkdir build
cd build
module purge
module load gcc/4.9.2 cuda/8.0
../configure --prefix=/projects/nlpl/software/glibc/2.18
make -j 8
make install
make localedata/install-locales
ln -s /usr/share/zoneinfo/Europe/Oslo /projects/nlpl/software/glibc/2.18/etc/localtime

To pre-configure the custom dynamic linker, allowing it to make use of ‘standard’ library locations that need not be on $LD_LIBRARY_PATH. In mid-September 2018, at least, basic CUDA libraries appear to be installed into /usr/lib64/, but only on GPU-enabled compute nodes.

qlogin --account=nn9106k --time=6:00:00 --partition=accel --gres=gpu:1
cp -pr /etc/ld.so.conf* /projects/nlpl/software/glibc/2.18/etc
/projects/nlpl/software/glibc/2.18/sbin/ldconfig 


Installation on Taito

wget https://ftp.gnu.org/gnu/glibc/glibc-2.18.tar.bz2
tar jpSxf glibc-2.18.tar.bz2
cd glibc-2.18
mkdir build
cd build
module purge
module load gcc/4.9.3
../configure --prefix=/proj/nlpl/software/glibc/2.18
make -j 8
make install
make localedata/install-locales
ln -s /usr/share/zoneinfo/Europe/Oslo /proj/nlpl/software/glibc/2.18/etc/localtime

To pre-configure the custom dynamic linker, allowing it to make use of ‘standard’ library locations that need not be on $LD_LIBRARY_PATH.

cp -pr /etc/ld.so.conf* /proj/nlpl/software/glibc/2.18/etc
echo "/lib64" >> /proj/nlpl/software/glibc/2.18/etc/ld.so.conf
/proj/nlpl/software/glibc/2.18/sbin/ldconfig 
Personal tools
Namespaces

Variants
Actions
Navigation
Tools