Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add toolchain definition for Fujitsu ARM toolchain #3677

Merged
merged 8 commits into from
May 25, 2021

Conversation

migueldiascosta
Copy link
Member

being tested at Fugaku

@migueldiascosta migueldiascosta changed the title add toolchain definition for Fujitsu ARM toolchain (WIP) add toolchain definition for Fujitsu ARM toolchain May 17, 2021
easybuild/toolchains/ffmpi.py Outdated Show resolved Hide resolved
easybuild/toolchains/ffmpi.py Outdated Show resolved Hide resolved
easybuild/toolchains/compiler/fujitsu.py Outdated Show resolved Hide resolved
easybuild/toolchains/linalg/fujitsussl.py Outdated Show resolved Hide resolved
easybuild/toolchains/mpi/fujitsumpi.py Outdated Show resolved Hide resolved
@migueldiascosta
Copy link
Member Author

@boegel see easybuilders/easybuild-easyconfigs#12863 (comment) about the hierarchy and the naming of ffmpi, do you have a better suggestion? (e.g. FujitsuCore? FujitsuCoreMPI?)

basically, this will be the toolchain used for most of what's in both GCCcore and gompi...

@boegel
Copy link
Member

boegel commented May 19, 2021

I replied in that issue, but to be clear: the Ffmpi class name is just an internal thing, and sort of separate with the actual toolchain name (but it's better to keep those in sync, of course).

'openmp': 'Kopenmp',
'unroll': 'funroll-loops',
'strict': ['Kfp_precision'],
'precise': ['Kfp_precision'],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apparently -Kfp_precision doesn't work in clang mode, closest we can get seems to be -Knoeval,nofast_matmul,nofp_contract,nofp_relaxed,noilfunc (and at least the tests that require precise in GMP do pass with these flags)

"""Setting FujitsuSSL specific SCALAPACK related variables"""
super(FujitsuSSL, self)._set_scalapack_variables()
for flags_var, _ in COMPILER_FLAGS:
self.variables.nappend(flags_var, ['SCALAPACK'])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-SSL2/-SSL2BLAMP and -SCALAPACK are only meant to be used when linking and generate a warning on compiling that is harmless but pollutes the logs

easybuild prepends -L to variables appended to LDFLAGS, so some refactoring seems to be necessary (?) in order to do this properly (is it worth it?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@migueldiascosta If you think it's important enough, please open an issue to follow up on this.


# enable clang compatibility mode
self.variables.nappend('CFLAGS', ['Nclang'])
self.variables.nappend('CXXFLAGS', ['Nclang'])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

turns out having -Nclang only in these flags messes up with configure scripts that probe for the compiler without them, and so detect the traditional mode instead (hitting this with the libffi included in _ctypes in Python 2.7.18)

one solution that seems to work is to move them to COMPILER_CC and COMPILER_CCX above (but I think then it's crucial to always enclose $CC etc. in double quotes, since they will include a space?)

'defaultprec': [],
'loose': ['Kfp_relaxed'],
'veryloose': ['Kfp_relaxed'],
'vectorize': {False: 'KNOSVE', True: 'KSVE'},
Copy link
Member Author

@migueldiascosta migueldiascosta May 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above, these don't seem to work in clang mode... SVE is set by default, so the alternative seems to be -Knosimd to disable...

@migueldiascosta
Copy link
Member Author

migueldiascosta commented May 22, 2021

I replied in that issue, but to be clear: the Ffmpi class name is just an internal thing, and sort of separate with the actual toolchain name (but it's better to keep those in sync, of course).

@boegel well, besides the name there is also the hierarchy, the reason I used only two levels is that the module provides compiler, mpi and linalg;

but even though they are there, easybuild doesn't have to enable all at once, e.g. we can have an additional non-mpi level "below" Ffmpi and/or move FujitsuSSL to the full toolchain level, would that be clearer?

DEFAULT_OPT_LEVEL: 'O2 -Kfast',
'lowopt': 'O1',
'noopt': 'O0',
'opt': 'O3 -Kfast',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-Kfast is actually -O3 plus a bunch of other stuff, so planning to play it safe and change to:

    DEFAULT_OPT_LEVEL: 'O2',
    'opt': 'Kfast',

In addition to the possibility of using 'opt': True in easyconfigs, -Kfast will be explicitly enabled in some easyblocks, e.g. fftw, berkeleygw (, fortranpythonpackage?)

@migueldiascosta
Copy link
Member Author

migueldiascosta commented May 23, 2021

but even though they are there, easybuild doesn't have to enable all at once, e.g. we can have an additional non-mpi level "below" Ffmpi and/or move FujitsuSSL to the full toolchain level, would that be clearer?

this does seem clearer (and cleaner), done in migueldiascosta@c095190

(at the time of writing, the commit doesn't show up in this PR, did I mess up or is GitHub acting up? will check back later)

@migueldiascosta
Copy link
Member Author

(weird, commit c095190 only showed up here after I pushed 558487e)

@boegel
Copy link
Member

boegel commented May 25, 2021

@migueldiascosta I'll assume this is fully functional in the current state (can't check myself, because I don't have access to Fugaku), so I'll go ahead and merge this, mostly because I want to ensure that the changes to easybuild/toolchains/fft/fftw.py don't have any negative side effects as we're ramping up to the extensive regression test for EasyBuild v4.4.0.

Follow-up PRs to fix specific issues are welcome, of course! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants