Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make sure that all easyconfig parameters are available for alternative module naming schemes #687

Closed
boegel opened this issue Sep 11, 2013 · 6 comments

Comments

@boegel
Copy link
Member

boegel commented Sep 11, 2013

Currently, only the easyconfig parameters that are available from dependency specifications are available for (alternative) module naming schemes (see also #679):

  • name
  • version
  • versionsuffix
  • toolchain

In order to make all easyconfig parameter available in a consistent way (i.e., also if only the dependency specification is available), we need to pass parsed easyconfigs to the module naming scheme function det_full_module_name.

For dependencies, we will thus need to implement the following steps:

  • try to generate module name from just the dependency specification; if that works, no other parameters other than the ones currently available (see list above) are required
  • if that fails, we need a parsed easyconfig file
    • the first attempt should be to try and locate the easyconfig file that is installed together with the module that resolves the dependency; the problem here is that we don't know the module name yet (that's exactly what we're trying to generate), so we need to generate an incomplete module name of some kind that can help us to do a directed search for the easyconfig file
    • if no installed module is found that matches the dependency specification, we need to resort to easyconfig files that are available in the robot search path(s)
  • if a matching easyconfig file was found, parse it and generate the module name with it (which should work, otherwise there a bug in the module naming scheme implementation)
  • if no matching easyconfig file was found, we're out of luck, and need to report that the module name could not be generated due to lack of information
@boegel
Copy link
Member Author

boegel commented Sep 17, 2013

We should also provide a bunch of useful functions in the easybuild.tools.module_naming_scheme namespace that make implementing custom module naming schemes easier, e.g.:

  • determine compiler used in toolchain
  • determine MPI lib used in toolchain
  • ...

@boegel
Copy link
Member Author

boegel commented Sep 17, 2013

Together with the utility functions, we should also implement a couple of examples of alternative module naming schemes, preferably existing ones (see #173 for a list).

@fgeorgatos
Copy link
Collaborator

live from hackathon, we'd like to address:

  1. moduleclass should be part of the naming scheme, we want it
  2. we should allow multiple active naming schemes in parallel (provide the flat namespace by default, in any case)

@fgeorgatos
Copy link
Collaborator

addendum:
we may also need to handle things like flavours of style "bits=32", "type=serial", "microarch=sandybridge` etc;
ref. http://sourceforge.net/p/flavours/wiki/Home/

any other better way than baptizing toolchains like clang/*-32bit-avx for -march=x86 -mcpu=corei7-avx?

This will become esp. important when people explore the big combinatorial space of builds which is now tractable!

@fgeorgatos
Copy link
Collaborator

if things like the following would be possible, that would be nice:

  • eb/1.9/toolchain/name/version
  • HPCBIOS/XYZ/toolchain/name/version # ie. just a tag, fed automagically via command-line or var

The whole point is about being able to implement any arbitrary Hierarchical Naming Scheme,
and ride nicely on Lmod's feature, to swap across buildsets; that would be seriously interesting service:
http://www.admin-magazine.com/HPC/Articles/Lmod-Alternative-Environment-Modules
(think of jumping across WRF-over-HDF-with-szlib-bug to WRF-over-HDF-without-szlib-bug)

It is not certain which approach is best, but sites/people should have the freedom to experiment
(a flexible generic naming scheme should make possible most permutations of a namespace)

@boegel
Copy link
Member Author

boegel commented Aug 8, 2014

fixed by #879 and #953

@boegel boegel closed this as completed Aug 8, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants