Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix make distcheck problem in dap4_test (but turning off nczarr_test/run_scalar.sh due to issue) #2546

Closed
wants to merge 10 commits into from
Closed

fix make distcheck problem in dap4_test (but turning off nczarr_test/run_scalar.sh due to issue) #2546

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
86cef5a
fixing baseline subdir problem
edwardhartnett Nov 4, 2022
10cf365
more makefile fixes for make distcheck
edwardhartnett Nov 4, 2022
23b247d
turned off nczarr_test/run_scalar.sh test
edwardhartnett Nov 4, 2022
cc7cf33
turning on distcheck in CI
edwardhartnett Nov 4, 2022
c71a748
turning off distcheck in CI
edwardhartnett Nov 4, 2022
81f922f
turning on distcheck in CI
edwardhartnett Nov 4, 2022
46bd409
turning on distcheck in CI
edwardhartnett Nov 4, 2022
8955433
turning on distcheck in CI
edwardhartnett Nov 4, 2022
3446e92
fixing docs missing EXTRA_DIST files
edwardhartnett Nov 4, 2022
07b0e47
turn on distcheck in CI for parallel builds
edwardhartnett Nov 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 3 additions & 14 deletions .github/workflows/run_tests_ubuntu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,14 +181,9 @@ jobs:
run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j
if: ${{ success() }}

- name: Build Tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of the DISTCHECK_CONFIGURE_FLAGS is great information, thanks! We'll want to keep the organization we have where build tests and run tests are separate steps in the workflow, for consistency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I'm not sure how that can be done with the distcheck target.

The distcheck target makes a dist, unpacks it, builds it (outside the source directory) and runs make check, all in one step.

So I don't know how we can build tests in one step and run them in another...

Copy link
Contributor

@DWesl DWesl Nov 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main use of the distcheck target over the check and installcheck targets, in my understanding, is checking whether the project

  1. Builds from a source distribution, not just from a repository
  2. Can perform behaviors expected of Autotools packages, namely an out-of-tree build.
  3. uninstalls and cleans up properly (make uninstall and make distclean).

If you can add a single run of make distcheck that only runs if all the Autotools runs of make check succeed, that should complete all objectives.

Another way to complete the second objective within the current framework is to perform all of the build and test steps in a subdirectory (i.e., start the configure step with mkdir build; cd build, and start every subsequent step with cd build).

It would also be possible to start the Linux Autotools runs with make dist, upload the resulting distributions, and use those instead of actions/checkout for getting the source code in later steps, which would achieve the first objective.

The make distclean check is easy with the out-of-tree build (if there's any files there, it's not in the state it was before ../configure) and should be straightforward with an in-repository build (fiddle with git status and friends until it tells you if there are any files not checked in, then check that the only new files are configure, Makefile.in, and friends).

shell: bash -l {0}
run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check TESTS="" -j
if: ${{ success() }}

- name: Run Tests
- name: distcheck
shell: bash -l {0}
run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make check -j
run: DISTCHECK_CONFIGURE_FLAGS="--disable-dap-remote-tests --enable-hdf4 --enable-doxygen --enable-external-server-tests" CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} make -j distcheck
if: ${{ success() }}

##
Expand Down Expand Up @@ -259,15 +254,9 @@ jobs:

- name: Build Tests
shell: bash -l {0}
run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} CC=mpicc make check TESTS="" -j
if: ${{ success() }}

- name: Run Tests
shell: bash -l {0}
run: CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} CC=mpicc make check -j
run: DISTCHECK_CONFIGURE_FLAGS="--enable-hdf4 --disable-dap-remote-tests --enable-parallel-tests --enable-pnetcdf" CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} CC=mpicc make -j distcheck
if: ${{ success() }}


#####
# One-Off CMake-based tests.
#####
Expand Down
5 changes: 5 additions & 0 deletions configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2146,6 +2146,11 @@ AC_CONFIG_FILES([Makefile
ncdap_test/expected3/Makefile
ncdap_test/expectremote3/Makefile
dap4_test/Makefile
dap4_test/baseline/Makefile
dap4_test/baselinethredds/Makefile
dap4_test/baselineraw/Makefile
dap4_test/baselineremote/Makefile
dap4_test/baselinehyrax/Makefile
plugins/Makefile
nczarr_test/Makefile
])
Expand Down
4 changes: 3 additions & 1 deletion dap4_test/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ EXTRA_DIST = test_parse.sh test_meta.sh test_data.sh \
test_raw.sh test_remote.sh test_hyrax.sh test_thredds.sh test_fillmismatch.sh \
tst_curlopt.sh d4test_common.sh \
daptestfiles dmrtestfiles cdltestfiles nctestfiles misctestfiles \
baseline baselineraw baselineremote baselinehyrax baselinethredds CMakeLists.txt test_common.h
CMakeLists.txt test_common.h

SUBDIRS = baseline baselinethredds baselineraw baselineremote baselinehyrax

CLEANFILES = *.exe
# This should only be left behind if using parallel io
Expand Down
56 changes: 56 additions & 0 deletions dap4_test/baseline/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
## This is a automake file, part of Unidata's netCDF package.
# Copyright 2018v, see the COPYRIGHT file for more information.

# This ensures that a bunch of DAP test files are present in the build.

# Ed Hartnett, 11/4/22

EXTRA_DIST = CMakeLists.txt test_atomic_array.nc.d4m \
test_enum_array.nc.d4m test_one_vararray.nc.d4m \
test_sequence_2.syn.d4m test_struct_nested.nc.d4m test_vlen3.nc.d4m \
Makefile.am test_atomic_array.nc.d4p test_enum_array.nc.d4p \
test_one_vararray.nc.d4p test_sequence_2.syn.d4p \
test_struct_nested.nc.d4p test_vlen3.nc.d4p test_anon_dim.2.syn.d4d \
test_atomic_array.syn.d4d test_enum.nc.d4d test_one_var.nc.d4d \
test_struct1.nc.d4d test_struct_type.nc.d4d test_vlen4.nc.d4d \
test_anon_dim.2.syn.d4m test_atomic_array.syn.d4m test_enum.nc.d4m \
test_one_var.nc.d4m test_struct1.nc.d4m test_struct_type.nc.d4m \
test_vlen4.nc.d4m test_anon_dim.2.syn.d4p test_atomic_array.syn.d4p \
test_enum.nc.d4p test_one_var.nc.d4p test_struct1.nc.d4p \
test_struct_type.nc.d4p test_vlen4.nc.d4p test_anon_dim.syn.d4d \
test_atomic_types.nc.d4d test_fill.nc.d4d test_opaque_array.7.nc.d4d \
test_struct_array.6.nc.d4d test_unlim1.nc.d4d test_vlen5.nc.d4d \
test_anon_dim.syn.d4m test_atomic_types.nc.d4m test_fill.nc.d4m \
test_opaque_array.7.nc.d4m test_struct_array.6.nc.d4m \
test_unlim1.nc.d4m test_vlen5.nc.d4m test_anon_dim.syn.d4p \
test_atomic_types.nc.d4p test_fill.nc.d4p test_opaque_array.7.nc.d4p \
test_struct_array.6.nc.d4p test_unlim1.nc.d4p test_vlen5.nc.d4p \
test_atomic_array.5.nc.d4d test_atomic_types.syn.d4d \
test_groups1.nc.d4d test_opaque_array.nc.d4d test_struct_array.nc.d4d \
test_utf8.nc.d4d test_vlen6.nc.d4d test_atomic_array.5.nc.d4m \
test_atomic_types.syn.d4m test_groups1.nc.d4m \
test_opaque_array.nc.d4m test_struct_array.nc.d4m test_utf8.nc.d4m \
test_vlen6.nc.d4m test_atomic_array.5.nc.d4p \
test_atomic_types.syn.d4p test_groups1.nc.d4p \
test_opaque_array.nc.d4p test_struct_array.nc.d4p test_utf8.nc.d4p \
test_vlen6.nc.d4p test_atomic_array.8.nc.d4d test_enum_2.nc.d4d \
test_one_vararray.1.nc.d4d test_opaque.nc.d4d \
test_struct_array.syn.d4d test_vlen1.nc.d4d test_vlen7.nc.d4d \
test_atomic_array.8.nc.d4m test_enum_2.nc.d4m \
test_one_vararray.1.nc.d4m test_opaque.nc.d4m \
test_struct_array.syn.d4m test_vlen1.nc.d4m test_vlen7.nc.d4m \
test_atomic_array.8.nc.d4p test_enum_2.nc.d4p \
test_one_vararray.1.nc.d4p test_opaque.nc.d4p \
test_struct_array.syn.d4p test_vlen1.nc.d4p test_vlen7.nc.d4p \
test_atomic_array.9.nc.d4d test_enum_array.4.nc.d4d \
test_one_vararray.3.nc.d4d test_sequence_1.syn.d4d \
test_struct_nested3.nc.d4d test_vlen2.nc.d4d test_vlen8.nc.d4d \
test_atomic_array.9.nc.d4m test_enum_array.4.nc.d4m \
test_one_vararray.3.nc.d4m test_sequence_1.syn.d4m \
test_struct_nested3.nc.d4m test_vlen2.nc.d4m test_vlen8.nc.d4m \
test_atomic_array.9.nc.d4p test_enum_array.4.nc.d4p \
test_one_vararray.3.nc.d4p test_sequence_1.syn.d4p \
test_struct_nested3.nc.d4p test_vlen2.nc.d4p test_vlen8.nc.d4p \
test_atomic_array.nc.d4d test_enum_array.nc.d4d \
test_one_vararray.nc.d4d test_sequence_2.syn.d4d \
test_struct_nested.nc.d4d test_vlen3.nc.d4d
10 changes: 10 additions & 0 deletions dap4_test/baselinehyrax/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## This is a automake file, part of Unidata's netCDF package.
# Copyright 2018v, see the COPYRIGHT file for more information.

# This ensures that a bunch of DAP test files are present in the build.

# Ed Hartnett, 11/4/22

EXTRA_DIST = amsre_20060131v5.dat.hyrax CMakeLists.txt \
nc4_nc_classic_comp.nc.hyrax nc4_strings_comp.nc.hyrax \
nc4_unsigned_types_comp.nc.hyrax ref_tst_compounds.nc.hyrax
23 changes: 23 additions & 0 deletions dap4_test/baselineraw/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## This is a automake file, part of Unidata's netCDF package.
# Copyright 2018v, see the COPYRIGHT file for more information.

# This ensures that a bunch of DAP test files are present in the build.

# Ed Hartnett, 11/4/22

EXTRA_DIST = CMakeLists.txt test_atomic_array.nc.dmp test_enum.nc.dmp \
test_one_var.nc.dmp test_struct_array.6.nc.dmp test_utf8.nc.dmp \
test_vlen7.nc.dmp test_atomic_array.syn.dmp test_fillmismatch.nc.dmp \
test_opaque_array.7.nc.dmp test_struct_array.nc.dmp test_vlen1.nc.dmp \
test_vlen8.nc.dmp test_anon_dim.2.syn.dmp test_atomic_types.nc.dmp \
test_fill.nc.dmp test_opaque_array.nc.dmp test_struct_array.syn.dmp \
test_vlen2.nc.dmp test_anon_dim.syn.dmp test_atomic_types.syn.dmp \
test_groups1.nc.dmp test_opaque.nc.dmp test_struct_nested3.nc.dmp \
test_vlen3.nc.dmp test_atomic_array.5.nc.dmp test_enum_2.nc.dmp \
test_one_vararray.1.nc.dmp test_sequence_1.syn.dmp \
test_struct_nested.nc.dmp test_vlen4.nc.dmp \
test_atomic_array.8.nc.dmp test_enum_array.4.nc.dmp \
test_one_vararray.3.nc.dmp test_sequence_2.syn.dmp \
test_struct_type.nc.dmp test_vlen5.nc.dmp test_atomic_array.9.nc.dmp \
test_enum_array.nc.dmp test_one_vararray.nc.dmp test_struct1.nc.dmp \
test_unlim1.nc.dmp test_vlen6.nc.dmp
22 changes: 22 additions & 0 deletions dap4_test/baselineremote/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## This is a automake file, part of Unidata's netCDF package.
# Copyright 2018v, see the COPYRIGHT file for more information.

# This ensures that a bunch of DAP test files are present in the build.

# Ed Hartnett, 11/4/22

EXTRA_DIST = CMakeLists.txt test_atomic_types.nc.dmp \
test_groups1.nc.dmp test_sequence_1.syn.dmp \
test_struct_nested3.nc.dmp test_vlen11.nc.dmp test_vlen4.nc.dmp \
test_vlen9.nc.dmp test_atomic_types.syn.dmp test_misc1.nc.dmp \
test_sequence_2.syn.dmp test_struct_nested.hdf5.dmp test_vlen1.nc.dmp \
test_vlen5.hdf5.dmp tst_fills.nc.dmp nc4_unsigned_types.nc.hyrax \
test_enum_2.nc.dmp test_one_vararray.nc.dmp test_struct1.nc.dmp \
test_struct_nested.nc.dmp test_vlen2.nc.dmp test_vlen5.nc.dmp \
test_anon_dim.syn.dmp test_enum_array.nc.dmp test_one_var.nc.dmp \
test_struct_array.nc.dmp test_struct_type.nc.dmp test_vlen3.hdf5.dmp \
test_vlen6.nc.dmp test_atomic_array.nc.dmp test_enum.nc.dmp \
test_opaque_array.nc.dmp test_struct_array.syn.dmp test_utf8.nc.dmp \
test_vlen3.nc.dmp test_vlen7.nc.dmp test_atomic_array.syn.dmp \
test_fill.nc.dmp test_opaque.nc.dmp test_struct_nested3.hdf5.dmp \
test_vlen10.nc.dmp test_vlen4.hdf5.dmp test_vlen8.nc.dmp
8 changes: 8 additions & 0 deletions dap4_test/baselinethredds/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## This is a automake file, part of Unidata's netCDF package.
# Copyright 2018v, see the COPYRIGHT file for more information.

# This ensures that a bunch of DAP test files are present in the build.

# Ed Hartnett, 11/4/22

EXTRA_DIST = CMakeLists.txt GOES16_TEST1.nc4.thredds
7 changes: 4 additions & 3 deletions docs/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,12 @@
# These files will be included with the dist.
EXTRA_DIST = netcdf.m4 DoxygenLayout.xml Doxyfile.in footer.html \
mainpage.dox tutorial.dox architecture.dox internal.dox \
windows-binaries.md dispatch.md building-with-cmake.md CMakeLists.txt groups.dox \
notes.md install-fortran.md credits.md auth.md filters.md \
windows-binaries.md dispatch.md building-with-cmake.md CMakeLists.txt \
groups.dox notes.md install-fortran.md credits.md auth.md filters.md \
obsolete/fan_utils.html indexing.dox inmemory.md FAQ.md \
known_problems.md COPYRIGHT.md inmeminternal.dox testserver.dox \
byterange.dox nczarr.md quantize.md all-error-codes.md
byterange.dox nczarr.md quantize.md all-error-codes.md \
attribute_conventions.md file_format_specifications.md internal.md

# Turn off parallel builds in this directory.
.NOTPARALLEL:
Expand Down
4 changes: 2 additions & 2 deletions nczarr_test/Makefile.am
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ TESTS += run_misc.sh
TESTS += run_nczarr_fill.sh
TESTS += run_jsonconvention.sh
TESTS += run_strings.sh
TESTS += run_scalar.sh
#TESTS += run_scalar.sh
TESTS += run_nulls.sh
endif

Expand Down Expand Up @@ -132,7 +132,7 @@ run_purezarr.sh run_interop.sh run_misc.sh \
run_filter.sh \
run_newformat.sh run_nczarr_fill.sh run_quantize.sh \
run_jsonconvention.sh run_nczfilter.sh run_unknown.sh \
run_scalar.sh run_strings.sh run_nulls.sh
run_scalar.sh run_strings.sh run_nulls.sh

EXTRA_DIST += \
ref_ut_map_create.cdl ref_ut_map_writedata.cdl ref_ut_map_writemeta2.cdl ref_ut_map_writemeta.cdl \
Expand Down
2 changes: 1 addition & 1 deletion nczarr_test/run_scalar.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ if test "x$srcdir" = x ; then srcdir=`pwd`; fi

# This shell script tests support for the NC_STRING type

set -e
set -e -x

zarrscalar() {
rm -f $2
Expand Down