Eosc/nuggets - Revision history

Oe: /* Accounting and Allocations */

2020-09-09T06:51:24Z

‎Accounting and Allocations

Oe: /* Software Management */

2020-09-09T06:48:58Z

‎Software Management

Oe: /* Accounting and Allocations */

2020-01-28T17:47:14Z

‎Accounting and Allocations

Oe: /* Software Management */

2020-01-28T17:41:52Z

‎Software Management

Oe: /* Module Usage Statistics */

2020-01-28T17:36:20Z

‎Module Usage Statistics

Oe: /* File System Management */

2020-01-28T17:33:07Z

‎File System Management

Oe: /* Software Management */

2020-01-11T23:33:30Z

‎Software Management

Oe: /* User Management */

2020-01-10T12:42:44Z

‎User Management

Oe: /* File System Management */

2020-01-10T11:01:05Z

‎File System Management

Oe: Created page with "= Background = Smaller, tangible activities to be implemented under the EOSC-Nordic umbrella, in support of the NLPL virtual laboratory. = User Management = The project ma..."

2020-01-10T10:57:23Z

Created page with "= Background = Smaller, tangible activities to be implemented under the EOSC-Nordic umbrella, in support of the NLPL virtual laboratory. = User Management = The project ma..."

New page

= Background =

Smaller, tangible activities to be implemented under the EOSC-Nordic umbrella,
in support of the NLPL virtual laboratory.

= User Management =

The project maintains an email list for all users with access to the NLPL virtual laboratory.
Need to extract programmatically, on Saga and Puhti, which users are associated with
NLPL billing project(s); run as cron(8) job every night (and then automatically update
the set of mailing list subscribers).

= File System Management =

= Software Management =

= Accounting and Allocations =

← Older revision		Revision as of 06:51, 9 September 2020
Line 54:		Line 54:

	At present, there appears to be no systematic distinction for accounting		At present, there appears to be no systematic distinction for accounting
−	purposes between cpu vs. gpu usage.	+	purposes between cpu vs. gpu usage (at least not in Norway).
		+
		+
		+	= Software Management =
		+
		+	We are still struggling to provide fully uniform software environments
		+	for NLPL users on Puhti and Saga (the current two instances of the
		+	virtual laboratory).
		+	Assuming we continue to build on the software modules infrastructure,
		+	two challenges are: (a) the inventory of basic building blocks differs
		+	(e.g. compilers, Python, CUDA, MPI) and (b) installation of add-on
		+	modules is only partially automated.
		+
		+	Could one agree with CSC and Sigma2 on a standardized inventory of
		+	basic modules and a common mechanism to build on top of that, e.g.
		+	either EasyBuild or Spack?
		+	since the spring of 2020, USIT and UiO-Inf have started to
		+	[http://wiki.nlpl.eu/index.php/Eosc/easybuild evaluate EasyBuild]
		+	for that purpose.

@@ Line 30: / Line 30: @@
 software/modules/                 chmod g+rwX,o+rX
 </pre>
 = Module Usage Statistics =

← Older revision		Revision as of 17:47, 28 January 2020
Line 54:		Line 54:

	= Accounting and Allocations =		= Accounting and Allocations =
		+
		+	Each country has its own mechanisms for user management, allocations,
		+	and accounting.
		+	In Norway, currently all NLPL users are organized in a single project
		+	(for allocation), making it difficult for the project manager to keep
		+	track of users, including their affiliations and period of association
		+	with the project.
		+	Project association and access to accounting information in both
		+	Finland and Norway is through in-house web services.
		+	Could one imagine building an API-like abstraction over the national
		+	services, e.g. a RESTful interface to retrieve usage statistics for a
		+	project or individual user in a specific period?
		+
		+	At present, there appears to be no systematic distinction for accounting
		+	purposes between cpu vs. gpu usage.

← Older revision		Revision as of 17:41, 28 January 2020
Line 33:		Line 33:
	= Software Management =		= Software Management =

		+	We are still struggling to provide fully uniform software environments
		+	for NLPL users on Puhti and Saga (the current two instances of the
		+	virtual laboratory).
		+	Assuming we continue to build on the software modules infrastructure,
		+	two challenges are: (a) the inventory of basic building blocks differs
		+	(e.g. compilers, Python, CUDA, MPI) and (b) installation of add-on
		+	modules is only partially automated.
		+
		+	Could one agree with CSC and Sigma2 on a standardized inventory of
		+	basic modules and a common mechanism to build on top of that, e.g.
		+	either EasyBuild or Spack?

	= Module Usage Statistics =		= Module Usage Statistics =

← Older revision		Revision as of 17:36, 28 January 2020
Line 35:		Line 35:

	= Module Usage Statistics =		= Module Usage Statistics =
		+
		+	On Abel, we used to have a crude mechanism tracking the use of NLPL software modules, viz.
		+	generating a log entry every time a module definition was <code>load</code>ed.
		+	This was accomplished by calling out to the logger(1) utility.
		+	To generalize this approach, we would minimally need a mechanism for logging
		+	remotely to a central NLPL server.

	= Accounting and Allocations =		= Accounting and Allocations =

@@ Line 26: / Line 26: @@
 <pre>
-software/modules/etc/nlpl/foobar  rm
+software/modules/etc/nlpl/tmp     rm -r
 software/modules/                 chgrp -R nlpl
 software/modules/                 chmod g+rwX,o+rX

← Older revision		Revision as of 23:33, 11 January 2020
Line 34:		Line 34:


		+	= Module Usage Statistics =

	= Accounting and Allocations =		= Accounting and Allocations =

@@ Line 8: / Line 8: @@
 The project maintains an email list for all users with access to the NLPL virtual laboratory.
-Need to extract programmatically, on Saga and Puhti, which users are associated with
+We need to extract programmatically, on Saga and Puhti, which users are associated with
-NLPL billing project(s); run as cron(8) job every night (and then automatically update
+any of the NLPL billing project(s), which should be read from a local file.
 the set of mailing list subscribers).
 = File System Management =

← Older revision		Revision as of 11:01, 10 January 2020
Line 15:		Line 15:
	= File System Management =		= File System Management =

		+	With multiple users maintaining a shared collection of directories and files
		+	(aka the community directory, the shared software and data installations that make up the
		+	core of the virtual laboratory), we frequently run into permission challenges that
		+	at times are tedious to resolve or require administrator priviliges (which the NLPL
		+	developers cannot have).
		+	We would like to see an interface where a cron(8) job regularly reads a specification
		+	of permissions and such from a file (maintained by NLPL), quality-controls the contents,
		+	and then executes corresponding actions, e.g. something like

		+	<pre>
		+	software/modules/etc/nlpl/foobar rm
		+	software/modules/ chgrp -R nlpl
		+	software/modules/ chmod g+rwX,o+rX
		+	</pre>

	= Software Management =		= Software Management =