Problems compiling MET with gcc compiler

[diegocarrio]

Dear MET_help support team,

I am having troubles trying to compile MET in our machines. I am following the following tutorials:

• https://metplus-training.readthedocs.io/en/latest/modules/Environment/met_installation.html
• https://dtcenter.org/community-code/model-evaluation-tools-met/download

but I get the following error message:

#########################################################
mc3:~/MET_TOOL_v4/10.1.2> ./compile_MET_all.sh install_met_env.mc3
TEST_BASE = /home/diego/MET_TOOL_v4/10.1.2/
COMPILER = intel
MET_SUBDIR = /home/diego/MET_TOOL_v4/10.1.2//
MET_TARBALL = met-10.1.2.20220516.tar.gz
USE_MODULES = FALSE
LD_LIBRARY_PATH = /home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/vg5/diego/miniconda3//lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib

Compiling libraries into: /home/diego/MET_TOOL_v4/10.1.2//external_libs
USE_MODULES = FALSE
export CC=
export CXX=
export FC=
export F77=
export F90=

Using python version:
Python 3.8.3

Compiling GSL_2.1 at mié oct 19 11:50:29 CEST 2022
cd /home/diego/MET_TOOL_v4/10.1.2/external_libs/gsl/gsl-2.1
./configure --prefix=/home/diego/MET_TOOL_v4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling MET at mié oct 19 11:52:32 CEST 2022
MET Configuration settings...
export MET_FONT_DIR=/home/diego/MET_TOOL_v4/10.1.2//fonts
export MET_GRIB2CLIB=/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib
export MET_GSL=/home/diego/MET_TOOL_v4/10.1.2//external_libs
export MET_PYTHON_CC=-I/vg5/diego/miniconda3/include/python3.8 -I/vg5/diego/miniconda3/include/python3.8 -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem /vg5/diego/miniconda3/include -fdebug-prefix-map=/tmp/build/80754af9/python_1589913560097/work=/usr/local/src/conda/python-3.8.3 -fdebug-prefix-map=/home/diego/miniconda3=/usr/local/src/conda-prefix -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -flto -DNDEBUG -fwrapv -O3 -Wall
export MET_TARBALL=/home/diego/MET_TOOL_v4/10.1.2//tar_files/met-10.1.2.20220516.tar.gz
export MET_GRIB2CINC=/home/diego/MET_TOOL_v4/10.1.2//external_libs/include
export MET_NETCDF=/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib
export MET_PYTHON=/vg5/diego/miniconda3/
export MET_PYTHON_LD=-L/vg5/diego/miniconda3/lib/python3.8/config-3.8-x86_64-linux-gnu -L/vg5/diego/miniconda3/lib -lcrypt -lpthread -ldl -lutil -lrt -lm -lm
export MET_SUBDIR=/home/diego/MET_TOOL_v4/10.1.2//
export MET_BUFRLIB=/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib
LDFLAGS = -Wl,--disable-new-dtags -Wl,-rpath,/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/vg5/diego/miniconda3//lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib -Wl,-rpath,
cd /home/diego/MET_TOOL_v4/10.1.2/met-10.1.2
./configure --prefix=/home/diego/MET_TOOL_v4/10.1.2// --bindir=/home/diego/MET_TOOL_v4/10.1.2//bin BUFRLIB_NAME=-lbufr GRIB2CLIB_NAME=-lgrib2c --enable-grib2 --enable-python > configure.log 2>&1
configure returned with non-zero (1) status
#########################################################

Looking at the config.log I found the following specific errors:

#########################################################
configure:3788: $? = 0
configure:3777: gcc -v >&5
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion= 'Ubuntu 5.4.0-6ubuntu1~ 16.04.12' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)
configure:3788: $? = 0
configure:3777: gcc -V >&5

gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.

configure:3788: $? = 1
configure:3777: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
#########################################################

and also this one:

#########################################################
configure:3960: $? = 0
configure:3967: ./conftest
configure:3971: $? = 0
configure:3986: result: no
configure:3991: checking for suffix of object files
configure:4013: gcc -c -Wall -g conftest.c >&5
configure:4017: $? = 0
configure:4038: result: o
configure:4042: checking whether we are using the GNU C compiler
configure:4061: gcc -c -Wall -g conftest.c >&5
configure:4061: $? = 0
configure:4070: result: yes
configure:4079: checking whether gcc accepts -g
configure:4099: gcc -c -g conftest.c >&5
configure:4099: $? = 0
configure:4140: result: yes
configure:4157: checking for gcc option to accept ISO C89
configure:4220: gcc -c -Wall -g conftest.c >&5
configure:4220: $? = 0
configure:4233: result: none needed
configure:4258: checking whether gcc understands -c and -o together
configure:4280: gcc -c conftest.c -o conftest2.o
configure:4283: $? = 0
configure:4280: gcc -c conftest.c -o conftest2.o
configure:4283: $? = 0
configure:4295: result: yes
configure:4314: checking dependency style of gcc
configure:4425: result: gcc3
configure:4449: checking for gcc option to support OpenMP
configure:4464: gcc -o conftest -Wall -g -Wl,--disable-new-dtags -Wl,-rpath,/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib:/vg5/diego/miniconda3//lib:/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib -Wl,-rpath, conftest.c -lhdf5_hl -lhdf5 -lz >&5

conftest.c:13:2: error: unknown type name 'choke'
choke me
^

In file included from conftest.c:15:0:
/usr/lib/gcc/x86_64-linux-gnu/5/include/omp.h:35:1: error: expected '=', ',', ';', 'asm' or 'attribute' before 'typedef'
typedef struct

^

configure:4464: $? = 1
configure: failed program was:
| /* confdefs.h /
| #define PACKAGE_NAME "MET"
| #define PACKAGE_TARNAME "met"
| #define PACKAGE_VERSION "10.1.2"
| #define PACKAGE_STRING "MET 10.1.2"
| #define PACKAGE_BUGREPORT "https://github.com/dtcenter/METplus/discussions"
| #define PACKAGE_URL "http://dtcenter.org/community-code/model-evaluation-tools-met"
| #define PACKAGE "met"
| #define VERSION "10.1.2"
| / end confdefs.h. */
#########################################################

To provide you with more information, here I show you my installation file:

mc3:~/MET_TOOL_v4/10.1.2> cat install_met_env.mc3

#Find the directory this script is called from
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

#Required
#Directory that is the root of the compile
export TEST_BASE=$DIR/

#Required
#Compiler options = gnu, intel, ics, ips, PrgEnv-intel, or pgi
#Compiler+version can be used for machines using modules e.g. gnu_6.3.0
export COMPILER=intel

#Required
#Root directory for creating/untaring met source code - usually same as TEST_BASE
export MET_SUBDIR=${TEST_BASE}/

#Required
#The name of the met tarbal usually downloaded with version from dtcenter.org and includes a version
#example - met-9.0.1.20200423.tar.gz
#met.tar.gz is used for compiling from cloned github repo
export MET_TARBALL=met-10.1.2.20220516.tar.gz

#Required
#Specifiy if machine useds modules for loading software
export USE_MODULES=FALSE

#Root directory of your python install
export PYTHON_LOC="$(/vg5/diego/miniconda3/bin/python3.8-config --prefix)"

#Directory of your python executable
export MET_PYTHON=${PYTHON_LOC}/

#Python ldflags created using python2.7-config
export MET_PYTHON_LD="$(/vg5/diego/miniconda3/bin/python3.8-config --ldflags)"

#Python cflags created using python2.7-config
export MET_PYTHON_CC="$(/vg5/diego/miniconda3/bin/python3.8-config --cflags)"

#64 bit machine or not
export SET_D64BIT=FALSE

#General CFLAGS
export CFLAGS="-Wall -g"

#General CXXLAGS
export CXXFLAGS="-Wall -g"

#Normally should be omitted or set to TRUE - only used if building from github repo
export USE_MET_TAR_FILE=TRUE

#If you've already compiled these and don't need to compile them again, set the following
export EXTERNAL_LIBS=${TEST_BASE}/external_libs
export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib
export MET_GRIB2CINC=${EXTERNAL_LIBS}/include
export GRIB2CLIB_NAME=-lgrib2c
export MET_BUFRLIB=${EXTERNAL_LIBS}/lib
export BUFRLIB_NAME=-lbufr
export MET_NETCDF=${EXTERNAL_LIBS}/lib
#Also, don't forget to set the following options to zero within the compile script if you've already compiled those libraries as well: COMPILE_GSL, COMPILE_HDF, COMPILE_HDFEOS, COMPILE_CAIRO, COMPILE_FREETYPE
#########################################################

Could you help me to solve this issue?

Thanks in advance,

Best Regards,
Diego

[jprestop]

HI @diegocarrio.

I see you are having problems compiling MET. Thank you for all of this helpful information and for using our tutorials/documentation. It would be great to look at the entire config.log file. Could you please either attach it here or send it up to our FTP server using the directions posted in the How to Send Us Data Discussion (and then let me know that you have posted it)?

Offhand, I see that in the configuration file install_met_env.mc3, you have the compiler set to intel:

export COMPILER=intel

In the output from the script, I see that the following are not set:

export CC=
export CXX=
export FC=
export F77=
export F90=

And in the config.log file, it looks like the compiler being used is "gcc". That may or may not be problematic, if the script is expecting the Intel compiler, but gcc is being used, as there are directives in the compile script to do one thing if using Intel, in some cases, but another thing, if using GNU compilers.

Did all of the libraries install successfully in /home/diego/MET_TOOL_v4/10.1.2//external_libs/lib and /home/diego/MET_TOOL_v4/10.1.2//external_libs/include?

If you'd like to try again, changing

export COMPILER=intel

to

export COMPILER=gnu

(if you truly would like to compile with the GNU compilers and not intel), I would also suggest simplifying the MET_PYTHON_CC variable from this:

export MET_PYTHON_CC=-I/vg5/diego/miniconda3/include/python3.8 -I/vg5/diego/miniconda3/include/python3.8 -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem /vg5/diego/miniconda3/include -fdebug-prefix-map=/tmp/build/80754af9/python_1589913560097/work=/usr/local/src/conda/python-3.8.3 -fdebug-prefix-map=/home/diego/miniconda3=/usr/local/src/conda-prefix -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -flto -DNDEBUG -fwrapv -O3 -Wall

to

export MET_PYTHON_CC=-I/vg5/diego/miniconda3/include/python3.8

Unfortunately, sometimes running "python3.8-config --cflags" can provide information that is not really necessary and can interfere with the installation.

I look forward to taking a look at your existing or a new config.log file, which will hopefully shed some further light on what is going wrong.

[jprestop]

Hi @diegocarrio. Thank you for posting your files to our FTP server. The error that you listed above should actually be ok to ignore. I believe that the real problem lies here:

configure:12914: /usr/bin/gcc -o conftest -Wall -g -I/home/diego/MET_TOOL_v4/10.1.2//external_libs/include -L/home/diego/MET_TOOL_v4/10.1.2//external_libs/lib conftest.c  >&5
/usr/bin/ld: /tmp/ccjdHquC.o: in function `main':                                                                                                                                
/home/diego/MET_TOOL_v4/10.1.2/external_libs/netcdf/netcdf-cxx4-4.3.0/conftest.c:44: undefined reference to `nc_create'
collect2: error: ld returned 1 exit status

I believe I have a solution that will fix that problem. I have attached a new compile_MET_all.sh script called compile_MET_all_fix.sh (which I had to zip in order to attach). After downloading and unzipping, you may need to change the permissions to make it executable:

chmod 775 compile_MET_all_fix.sh

Please note that I changed line 666 from:

./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > configure.log 2>&1

to

./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS="-L${LIB_DIR}/lib -lhdf5_hl -lhdf5 -lz -lnetcdf"> configure.log 2>&1

to explicitly tell netCDF-cxx to link with the specified libraries.

Please try rerunning using the new script. If you'd like to avoid the recompilation of the libraries you already have installed, you can simply uncomment these lines in your instal_met_env.mc4 file:

Changing the following:

#export EXTERNAL_LIBS=${TEST_BASE}/external_libs
#export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib
#export MET_GRIB2CINC=${EXTERNAL_LIBS}/include
#export GRIB2CLIB_NAME=-lgrib2c
#export MET_BUFRLIB=${EXTERNAL_LIBS}/lib
#export BUFRLIB_NAME=-lbufr

to

export EXTERNAL_LIBS=${TEST_BASE}/external_libs
export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib
export MET_GRIB2CINC=${EXTERNAL_LIBS}/include
export GRIB2CLIB_NAME=-lgrib2c
export MET_BUFRLIB=${EXTERNAL_LIBS}/lib
export BUFRLIB_NAME=-lbufr

Please let us know how it goes.

compile_MET_all_fix.sh.zip

[diegocarrio]

Hi @jprestop, thank you very much for your help. I followed your instructions, but unfortunately I got the same error while compiling netcdf_cxx. I have uploaded the configure file log, the compile_*.sh and the install_met_env files again in case you find something else. However, looking more in detail, I checked the hdf5 compilation and seems to have the same problem, so I guess we need to fix hdf5 compilation first. I have uploaded to the ftp the log file for the hdf5 in the hdf5 folder. I hope it helps.

Thanks in advance,
Diego

[jprestop]

Hi @diegocarrio. You're welcome. We're happy to help.

I looked at the make_install_hdf5.log file, and it looked like hdf5 compiled successfully to me. However, to be sure and to check on all of the installed libraries could you please upload the output of the following commands to our ftp server?

ls /home/diego/MET_TOOL_v4/10.1.2/external_libs/lib
ls /home/diego/MET_TOOL_v4/10.1.2/external_libs/include

I did notice an issue in the make_install_hdf5.log file, which was:

/bin/bash: /vg5/diego/miniconda3//lib/libtinfo.so.6: no version information available (required by /bin/bash)

In googling, it looks like you may need/want to install ncurses. To install, it would look something like this:

conda install -c conda-forge ncurses

It looks like some of the errors were cleaned up in the new config.log file that you sent for the netcdf-cxx library, so I think you're making progress. However, I saw the following repeatedly:

/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_version_info@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_init@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_setopt@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_cleanup@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_perform@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_strerror@CURL_OPENSSL_4'
/usr/bin/ld: /home/diego/MET_TOOL_v5/10.1.2//external_libs/lib/libnetcdf.so: undefined reference to `curl_easy_getinfo@CURL_OPENSSL_4'

On the same line 666, that I previously modified in compile_MET_all.sh to create compile_MET_all_fix.sh, please add "-lcurl" preceded by "-LCURL_LIB" where "CURL_LIB" is the path to the library directory where libcurl is installed.

Then, please try rerunning compile_MET_all.sh again. Please let me know if you have any questions, and please let us know how it goes.

[diegocarrio]

Hi @jprestop,

Thank you for your kind email and comments. I really appreciate!
I do not have a lot of expertise in compiling this kind of programs, so apologies for my question that probably are very silly.

1. I have uploaded to the server the lib and include folders "zipped" as you requested.

2. As I am using the python3.8 in miniconda, I am not sure how to install the ncurse package... If I use the command you mentioned, the packages will be installed in the default python version of the system, and not in the miniconda. Am I right? I do not know how to install ncurse in the miniconda. Any idea?

3. I do not know how to find the path of the libcurl libraries you are mentioning.... In the miniconda folder, I found "/vg5/diego/miniconda3/lib/libcurl.so". Is this the path I should use? I should modify the line 666 as:

./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS="-L${LIB_DIR}/lib -lhdf5_hl -lhdf5 -lz -lnetcdf" LIBS="-L${CURL_LIB}/lib -lcurl"> configure.log 2>&1

where CURL_LIB="/vg5/diego/miniconda3/"

Thanks in advance,
Diego

[diegocarrio]

Dear @jprestop

I have added full directory 10.1.2.zip in the server, so you can see all I have there. Here I was using a python 3.8 from my miniconda installation, but I did not exactly how to install "ncurses" in miniconda... For this reason, I have tried another compilation using the python3.8 installed in the system (not on miniconda). Using this python version and avoiding miniconda it seems I have done some progress...

mc4:~/MET_TOOL_mc4/10.1.2> ./compile_MET_all_fix.sh install_met_env.mc4

TEST_BASE = /home/diego/MET_TOOL_mc4/10.1.2/
COMPILER = gnu
MET_SUBDIR = /home/diego/MET_TOOL_mc4/10.1.2//
MET_TARBALL = met-10.1.2.20220516.tar.gz
USE_MODULES = FALSE
LD_LIBRARY_PATH = /home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/usr//lib

Compiling libraries into: /home/diego/MET_TOOL_mc4/10.1.2//external_libs

USE_MODULES = FALSE

export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export FC=/usr/bin/gfortran
export F77=/usr/bin/gfortran
export F90=/usr/bin/gfortran

Using python version:
Python 3.8.10

Compiling GSL_2.1 at Sun Oct 23 12:04:52 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/gsl/gsl-2.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
^[[24~^[[24~make install > make_install.log 2>&1

Compiling BUFRLIB_v11.3.0 at Sun Oct 23 12:14:05 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/bufrlib/BUFRLIB_v11.3.0
endiantest.c:33:1: warning: return type defaults to 'int' [-Wimplicit-int]
33 | main()
| ^~~~

Compiling ZLIB at Sun Oct 23 12:14:23 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/zlib/zlib-1.2.11
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1
rm /home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib/zlib.a

Compiling LIBPNG at Sun Oct 23 12:14:33 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/libpng/libpng-1.6.37
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling JASPER at Sun Oct 23 12:15:32 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/jasper/jasper-1.900.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling G2CLIB at Sun Oct 23 12:16:57 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/g2clib/g2clib-1.6.0
make > make.log 2>&1

Compiling HDF5 at Sun Oct 23 12:17:19 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/hdf5/hdf5-1.8.18
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs --with-zlib=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fPIC LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling NetCDF-C at Sun Oct 23 12:29:56 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/netcdf/netcdf-4.4.1.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling NetCDF-CXX at Sun Oct 23 12:32:25 CEST 2022
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/netcdf/netcdf-cxx4-4.3.0
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling MET at Sun Oct 23 12:33:42 CEST 2022
MET Configuration settings...
export MET_PYTHON_CC=-I/usr/include/python3.8
export MET_NETCDF=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_GRIB2CLIB=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
export MET_TARBALL=/home/diego/MET_TOOL_mc4/10.1.2//tar_files/met-10.1.2.20220516.tar.gz
export MET_HDF5=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_FONT_DIR=/home/diego/MET_TOOL_mc4/10.1.2//fonts
export MET_GSL=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_BUFRLIB=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
export MET_GRIB2CINC=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include
export MET_SUBDIR=/home/diego/MET_TOOL_mc4/10.1.2//
export MET_PYTHON_LD=-L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib -lcrypt -lpthread -ldl -lutil -lm -lm
export MET_PYTHON=/usr/
LDFLAGS = -Wl,--disable-new-dtags -Wl,-rpath,/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/usr//lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -Wl,-rpath,/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
cd /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2// --bindir=/home/diego/MET_TOOL_mc4/10.1.2//bin BUFRLIB_NAME=-lbufr GRIB2CLIB_NAME=-lgrib2c --enable-grib2 --enable-python > configure.log 2>&1
make > make.log 2>&1

make returned with non-zero (2) status

Now the error I got in the netcdf-cxx4-4.3.0/make_install.log:

/bin/bash ../libtool --tag=CXX --mode=link /usr/bin/g++ -Wall -g -version-info 1:3:0 -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -o libnetcdf_c++4.la -rpath /home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib ncAtt.lo ncCheck.lo ncCompoundType.lo ncDim.lo ncEnumType.lo ncException.lo ncFile.lo ncGroup.lo ncGroupAtt.lo ncOpaqueType.lo ncType.lo ncVar.lo ncVarAtt.lo ncVlenType.lo ncByte.lo ncChar.lo ncShort.lo ncInt.lo ncFloat.lo ncDouble.lo ncUbyte.lo ncUshort.lo ncUint.lo ncInt64.lo ncUint64.lo ncString.lo -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -lhdf5_hl -lhdf5 -lz -lnetcdf -L -lcurl
libtool: link: require no space between -L' and -lcurl'
make[1]: *** [Makefile:693: libnetcdf_c++4.la] Error 1
make[1]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/external_libs/netcdf/netcdf-cxx4-4.3.0/cxx4'
make: *** [Makefile:516: install-recursive] Error 1

In the compile_MET_all_fix.sh I had the following modified 666 line:

export CURL_DIR="/usr/lib/x86_64-linux-gnu/"
./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS="-L${LIB_DIR}/lib -lhdf5_hl -lhdf5 -lz -lnetcdf -L${CURL_LIB} -lcurl"> configure.log 2>&1

In $CURL_DIR I have the following:

ls /usr/lib/x86_64-linux-gnu/libcurl*
/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.3 /usr/lib/x86_64-linux-gnu/libcurl.a /usr/lib/x86_64-linux-gnu/libcurl.so.4
/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 /usr/lib/x86_64-linux-gnu/libcurl.la /usr/lib/x86_64-linux-gnu/libcurl.so.4.6.0
/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.6.0 /usr/lib/x86_64-linux-gnu/libcurl.so

I have uploaded the entire compile folder in the ftp server as "MET_23October2022.zip". I hope it helps.

Thank you in advance,
Diego

[diegocarrio]

Hi @jprestop

I have modified the line 666 of the compile_MET_all_fix.sh in the following way and seems to work:

export CURL_DIR="/usr/lib/x86_64-linux-gnu/"

./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS="-L${LIB_DIR}/lib -lhdf5_hl -lhdf5 -lz -lnetcdf" CURL_LIB="-L${CURL_LIB} -lcurl" > configure.log 2>&1

Now, the netcdf libraries seem to finish successfully with the following message:

+-------------------------------------------------------------+
| Congratulations! You have successfully installed netCDF! |
| |
| You can use script "nc-config" to find out the relevant |
| compiler options to build your application. Enter |
| |
| nc-config --help |
| |
| for additional information. |
| |
| CAUTION: |
| |
| If you have not already run "make check", then we strongly |
| recommend you do so. It does not take very long. |
| |
| Before using netCDF to store important data, test your |
| build with "make check". |
| |
| NetCDF is tested nightly on many platforms at Unidata |
| but your platform is probably different in some ways. |
| |
| If any tests fail, please see the netCDF web site: |
| http://www.unidata.ucar.edu/software/netcdf/ |
| |
| NetCDF is developed and maintained at the Unidata Program |
| Center. Unidata provides a broad array of data and software |
| tools for use in geoscience education and research. |
| http://www.unidata.ucar.edu |
+-------------------------------------------------------------+

+-------------------------------------------------------------+
| Congratulations! You have successfully installed the |
| new netCDF-4 C++ Library |
| |
| CAUTION: |
| |
| If you have not already run "make check", then we strongly |
| recommend you do so. It does not take very long. |
| |
| Before using netCDF to store important data, test your |
| build with "make check". |
| |
| If any tests fail, please see the netCDF web site: |
| http://www.unidata.ucar.edu/software/netcdf/ |
| |
| NetCDF is developed and maintained at the Unidata Program |
| Center. Unidata provides a broad array of data and software |
| tools for use in geoscience education and research. |
| http://www.unidata.ucar.edu |
+-------------------------------------------------------------+

Now, the make.log file in met10.1.2 show me the following error:
.
.
.

/home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:172: undefined reference to PyList_Size' /usr/bin/ld: ../../../../src/basic/vx_util/libvx_util.a(libvx_util_a-python_line.o): in function PyLineDataFile::make_data_line()':
/home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:243: undefined reference to PyList_GetItem' /usr/bin/ld: /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:267: undefined reference to PyList_Size'
/usr/bin/ld: /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:282: undefined reference to PyList_GetItem' /usr/bin/ld: ../../../../src/basic/vx_util/libvx_util.a(libvx_util_a-python_line.o): in function PyLineDataFile::do_straight()':
/home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:335: undefined reference to PyUnicode_FromString' /usr/bin/ld: /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:335: undefined reference to PyImport_Import'
/usr/bin/ld: /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:337: undefined reference to PyErr_Occurred' /usr/bin/ld: /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/basic/vx_util/python_line.cc:339: undefined reference to PyErr_Print'
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:431: ensemble_stat] Error 1
make[5]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/tools/core/ensemble_stat'
make[4]: *** [Makefile:380: all-recursive] Error 1
make[4]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/tools/core'
make[3]: *** [Makefile:374: all-recursive] Error 1
make[3]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src/tools'
make[2]: *** [Makefile:371: all-recursive] Error 1
make[2]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/met-10.1.2/src'
make[1]: *** [Makefile:426: all-recursive] Error 1
make[1]: Leaving directory '/vg5/diego/MET_TOOL_mc4/10.1.2/met-10.1.2'

I have uploaded this file in the ftp server with the following name: "make.log_24Oct2022". I hope it can help!

Thanks in advance,
Diego

[jprestop]

Hi Diego. Thank you for sending your make.log file. Looking at the recent install_met_env.mc4 file that you sent, I see that you had the following set:

export MET_PYTHON_LD="$(/vg5/diego/miniconda3/bin/python3.8-config --ldflags)"

From the make.log file it looks like the output of "python3.8-config --ldflags" is not producing everything it needs to. For example, I see the following in the make.log file:

-L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib  -lcrypt -lpthread -ldl  -lutil -lm

However, that line is missing "-lpython3.8", which is necessary for the MET compilation using python embedding. In your install_met_env.mc4 file please change the line for MET_PYTHON_LD to the following:

export MET_PYTHON_LD="-L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib -lpython3.8 -lcrypt -lpthread -ldl  -lutil -lm"

and trying rerunning. I believe that will resolve the problem you are encountering. Please let us know how it goes.

[diegocarrio]

Dear @jprestop ,

Thank you very much for your comment. It seems that applying your suggestion solve the problem. This is the output I get from the compilation:

./compile_MET_all_fix.sh install_met_env.mc4

TEST_BASE = /home/diego/MET_TOOL_mc4/10.1.2/
COMPILER = gnu
MET_SUBDIR = /home/diego/MET_TOOL_mc4/10.1.2//
MET_TARBALL = met-10.1.2.20220516.tar.gz
USE_MODULES = FALSE
LD_LIBRARY_PATH = /home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/usr//lib

Compiling libraries into: /home/diego/MET_TOOL_mc4/10.1.2//external_libs

USE_MODULES = FALSE

export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export FC=/usr/bin/gfortran
export F77=/usr/bin/gfortran
export F90=/usr/bin/gfortran

Using python version:
Python 3.8.10

Compiling GSL_2.1 at mié 26 oct 2022 07:10:32 CEST

cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/gsl/gsl-2.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling BUFRLIB_v11.3.0 at mié 26 oct 2022 07:16:31 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/bufrlib/BUFRLIB_v11.3.0
endiantest.c:33:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
33 | main()
| ^~~~

Compiling ZLIB at mié 26 oct 2022 07:16:52 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/zlib/zlib-1.2.11
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1
rm /home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib/zlib.a

Compiling LIBPNG at mié 26 oct 2022 07:17:02 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/libpng/libpng-1.6.37
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling JASPER at mié 26 oct 2022 07:17:39 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/jasper/jasper-1.900.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1

Compiling G2CLIB at mié 26 oct 2022 07:18:48 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/g2clib/g2clib-1.6.0
make > make.log 2>&1

Compiling HDF5 at mié 26 oct 2022 07:19:00 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/hdf5/hdf5-1.8.18
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs --with-zlib=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fPIC LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling NetCDF-C at mié 26 oct 2022 07:26:17 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/netcdf/netcdf-4.4.1.1
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling NetCDF-CXX at mié 26 oct 2022 07:27:33 CEST
cd /home/diego/MET_TOOL_mc4/10.1.2/external_libs/netcdf/netcdf-cxx4-4.3.0
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2//external_libs LDFLAGS=-L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib CPPFLAGS=-I/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include > configure.log 2>&1
make install > make_install.log 2>&1

Compiling MET at mié 26 oct 2022 07:28:13 CEST
MET Configuration settings...
export MET_PYTHON_CC=-I/usr/include/python3.8
export MET_NETCDF=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_GRIB2CLIB=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
export MET_TARBALL=/home/diego/MET_TOOL_mc4/10.1.2//tar_files/met-10.1.2.20220516.tar.gz
export MET_HDF5=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_FONT_DIR=/home/diego/MET_TOOL_mc4/10.1.2//fonts
export MET_GSL=/home/diego/MET_TOOL_mc4/10.1.2//external_libs
export MET_BUFRLIB=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
export MET_GRIB2CINC=/home/diego/MET_TOOL_mc4/10.1.2//external_libs/include
export MET_SUBDIR=/home/diego/MET_TOOL_mc4/10.1.2//
export MET_PYTHON_LD=-L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib -lpython3.8 -lcrypt -lpthread -ldl -lutil -lm
export MET_PYTHON=/usr/
LDFLAGS = -Wl,--disable-new-dtags -Wl,-rpath,/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/usr//lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -Wl,-rpath,/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib:/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib -L/home/diego/MET_TOOL_mc4/10.1.2//external_libs/lib
cd /home/diego/MET_TOOL_mc4/10.1.2/met-10.1.2
./configure --prefix=/home/diego/MET_TOOL_mc4/10.1.2// --bindir=/home/diego/MET_TOOL_mc4/10.1.2//bin BUFRLIB_NAME=-lbufr GRIB2CLIB_NAME=-lgrib2c --enable-grib2 --enable-python > configure.log 2>&1
make > make.log 2>&1
make install > make_install.log 2>&1
make test > make_test.log 2>&1
Finished compiling at mié 26 oct 2022 07:43:01 CEST

Apparently I do not get any error!! I have look at the make_install.log, and I do not see the problems we get before. Now, I have a new log file called "make_test.log" that seems also to work well (I will upload to the server the "make_install.log" and "make_test.log" to be sure the compilation works as expected. I have renamed those files as "make_install.log_26Oct2022" and "make_test.log_26Oct2022", respectively.

It seems that the compilation works well because I get binary files that I was not getting before (see attached).

Let me know what do you think.

Thanks in advance,
Diego

[jprestop]

Hi @diegocarrio. This is great news! I took a look at the make_install.log file that you uploaded, checked for errors, and did not see any. Based on the screenshot as well, it does look like you got a successful compilation! That's awesome!

If you don't have any further questions, about the MET compilation, I’d like to mark this discussion as being answered and “lock it” to prevent future posts. That’s how we encourage users to create new discussions for new questions. But I wanted to give you an opportunity to comment on it before doing so. Please feel free to select one of the responses as being the best answer to your original question. That’ll help future users with similar questions find answers.

[diegocarrio]

Thank you very much for ALL your help @jprestop ! I am very happy to have a successful compilation! I really appreciate it!
I will try to run the code with some real data and I will create a new discussion if I have further questions.

Best Regards,
Diego

[jprestop]

You're very welcome! Sounds great! Thank you!