Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make testing more robust #381

Merged
merged 6 commits into from
Jul 17, 2023
Merged

Make testing more robust #381

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5b30e85
Shut down redis server even if tests fail; do so while still reportin…
billschereriii Jul 13, 2023
92d029d
Fix PR number
billschereriii Jul 13, 2023
15eb716
Revert change to test case
billschereriii Jul 13, 2023
3bc412d
Fix syntax error
billschereriii Jul 17, 2023
03179c6
Merge
billschereriii Jul 17, 2023
87f5efb
Add message for test failure
billschereriii Jul 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 22 additions & 16 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,9 +320,11 @@ define run_smartredis_tests_with_standalone_server
echo "Running standalone tests" && \
PYTHONFAULTHANDLER=1 python -m pytest $(SR_TEST_PYTEST_FLAGS) $(COV_FLAGS) \
$(SKIP_DOCKER) $(SKIP_PYTHON) $(SKIP_FORTRAN) \
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) && \
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) ; \
(testresult=$$?; \
echo "Shutting down standalone Redis server" && \
python utils/launch_redis.py --port $(SR_TEST_PORT) --nodes 1 --stop && \
test $$testresult -eq 0 || echo "Standalone tests failed"; exit $$testresult) && \
echo "Standalone tests complete"
endef

Expand All @@ -339,10 +341,12 @@ define run_smartredis_tests_with_clustered_server
echo "Running clustered tests" && \
PYTHONFAULTHANDLER=1 python -m pytest $(SR_TEST_PYTEST_FLAGS) $(COV_FLAGS) \
$(SKIP_DOCKER) $(SKIP_PYTHON) $(SKIP_FORTRAN) \
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) && \
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) ; \
(testresult=$$?; \
echo "Shutting down clustered Redis server" && \
python utils/launch_redis.py --port $(SR_TEST_PORT) \
--nodes $(SR_TEST_NODES) --stop && \
--nodes $(SR_TEST_NODES) --stop; \
test $$testresult -eq 0 || echo "Clustered tests failed"; exit $$testresult) && \
echo "Clustered tests complete"
endef

Expand All @@ -361,10 +365,12 @@ define run_smartredis_tests_with_uds_server
echo "Running standalone tests with Unix Domain Socket connection" && \
PYTHONFAULTHANDLER=1 python -m pytest $(SR_TEST_PYTEST_FLAGS) $(COV_FLAGS) \
$(SKIP_DOCKER) $(SKIP_PYTHON) $(SKIP_FORTRAN) \
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) && \
echo "Shutting down standalone Redis server with Unix Domain Socket support"
--build $(SR_BUILD) --sr_fortran $(SR_FORTRAN) $(1) ; \
(testresult=$$?; \
echo "Shutting down standalone Redis server with Unix Domain Socket support" && \
python utils/launch_redis.py --port $(SR_TEST_PORT) --nodes 1 \
--udsport $(SR_TEST_UDS_FILE) --stop && \
--udsport $(SR_TEST_UDS_FILE) --stop; \
test $$testresult -eq 0 || echo "UDS tests failed"; exit $$testresult) && \
echo "UDS tests complete"
endef

Expand Down Expand Up @@ -395,15 +401,15 @@ test: test-deps
test: build-tests
test: SR_TEST_PYTEST_FLAGS := -vv
test:
-@$(call run_smartredis_tests_with_server,./tests)
@$(call run_smartredis_tests_with_server,./tests)

# help: test-verbose - Build and run all tests [verbosely]
.PHONY: test-verbose
test-verbose: test-deps
test-verbose: build-tests
test-verbose: SR_TEST_PYTEST_FLAGS := -vv -s
test-verbose:
-@$(call run_smartredis_tests_with_server,./tests)
@$(call run_smartredis_tests_with_server,./tests)

# help: test-verbose-with-coverage - Build and run all tests [verbose-with-coverage]
.PHONY: test-verbose-with-coverage
Expand All @@ -412,15 +418,15 @@ test-verbose-with-coverage: test-deps
test-verbose-with-coverage: build-tests
test-verbose-with-coverage: SR_TEST_PYTEST_FLAGS := -vv -s
test-verbose-with-coverage:
-@$(call run_smartredis_tests_with_server,./tests)
@$(call run_smartredis_tests_with_server,./tests)

# help: test-c - Build and run all C tests
.PHONY: test-c
test-c: test-deps
test-c: build-test-c
test-c: SR_TEST_PYTEST_FLAGS := -vv -s
test-c:
-@$(call run_smartredis_tests_with_server,./tests/c)
@$(call run_smartredis_tests_with_server,./tests/c)

# help: test-cpp - Build and run all C++ tests
.PHONY: test-cpp
Expand All @@ -429,15 +435,15 @@ test-cpp: build-test-cpp
test-cpp: build-unit-test-cpp
test-cpp: SR_TEST_PYTEST_FLAGS := -vv -s
test-cpp:
-@$(call run_smartredis_tests_with_server,./tests/cpp)
@$(call run_smartredis_tests_with_server,./tests/cpp)

# help: unit-test-cpp - Build and run unit tests for C++
.PHONY: unit-test-cpp
unit-test-cpp: test-deps
unit-test-cpp: build-unit-test-cpp
unit-test-cpp: SR_TEST_PYTEST_FLAGS := -vv -s
unit-test-cpp:
-@$(call run_smartredis_tests_with_server,./tests/cpp/unit-tests)
@$(call run_smartredis_tests_with_server,./tests/cpp/unit-tests)

# help: test-py - run python tests
.PHONY: test-py
Expand All @@ -446,7 +452,7 @@ test-py: SR_PYTHON := ON
test-py: lib
test-py: SR_TEST_PYTEST_FLAGS := -vv
test-py:
-@$(call run_smartredis_tests_with_server,./tests/python)
@$(call run_smartredis_tests_with_server,./tests/python)

# help: test-fortran - run fortran tests
.PHONY: test-fortran
Expand All @@ -455,7 +461,7 @@ test-fortran: test-deps
test-fortran: build-test-fortran
test-fortran: SR_TEST_PYTEST_FLAGS := -vv
test-fortran:
-@$(call run_smartredis_tests_with_server,./tests/fortran)
@$(call run_smartredis_tests_with_server,./tests/fortran)

# help: testpy-cov - run python tests with coverage
.PHONY: testpy-cov
Expand All @@ -464,15 +470,15 @@ testpy-cov: SR_PYTHON := ON
testpy-cov: SR_TEST_PYTEST_FLAGS := -vv
testpy-cov: COV_FLAGS := --cov=./src/python/module/smartredis/
testpy-cov:
-@$(call run_smartredis_tests_with_server,./tests/python)
@$(call run_smartredis_tests_with_server,./tests/python)

# help: test-examples - Build and run all examples
.PHONY: test-examples
test-examples: test-deps
test-examples: build-examples
testpy-cov: SR_TEST_PYTEST_FLAGS := -vv -s
test-examples:
-@$(call run_smartredis_tests_with_server,./examples)
@$(call run_smartredis_tests_with_server,./examples)


############################################################################
Expand Down
4 changes: 4 additions & 0 deletions doc/changelog.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,22 @@ To be released at some future point in time

Description

- Improve robustness of test runs
- Fix installation link
- Update supported languages documentation
- Removed obsolete files
- Improved clustered redis initialization

Detailed Notes

- Preserve the shell output of test runs while making sure that server shutdown happens unconditionally (PR381_)
- Fix incorrect link to installation documentation (PR380_)
- Update language support matrix in documentation to reflect updates from the last release (PR379_)
- Fix typo causing startup failure in utility script for unit tests (PR378_)
- Deleted obsolete build and testing files that are no longer needed with the new build and test system (PR366_)
- Reuse existing redis connection when mapping the Redis cluster (PR364_)

.. _PR381: https://github.com/CrayLabs/SmartRedis/pull/381
.. _PR380: https://github.com/CrayLabs/SmartRedis/pull/380
.. _PR379: https://github.com/CrayLabs/SmartRedis/pull/379
.. _PR378: https://github.com/CrayLabs/SmartRedis/pull/378
Expand Down