Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: liveness tests #1456

Merged
merged 9 commits into from
Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 23 additions & 47 deletions .github/workflows/sims.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,29 +26,29 @@ jobs:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary

# test-sim-nondeterminism:
# runs-on: ubuntu-latest
# needs: Build
# steps:
# - uses: actions/[email protected]
# - uses: actions/[email protected]
# with:
# go-version: 1.17
# - uses: technote-space/[email protected]
# with:
# PATTERNS: |
# **/**.go
# go.mod
# go.sum
# - uses: actions/[email protected]
# with:
# path: ~/go/bin
# key: ${{ runner.os }}-go-runsim-binary
# if: "env.GIT_DIFF != ''"
# - name: test nondeterminism
# run: |
# make test-sim-nondeterminism
# if: "env.GIT_DIFF != ''"
# test-sim-nondeterminism:
# runs-on: ubuntu-latest
# needs: Build
# steps:
# - uses: actions/[email protected]
# - uses: actions/[email protected]
# with:
# go-version: 1.17
# - uses: technote-space/[email protected]
# with:
# PATTERNS: |
# **/**.go
# go.mod
# go.sum
# - uses: actions/[email protected]
# with:
# path: ~/go/bin
# key: ${{ runner.os }}-go-runsim-binary
# if: "env.GIT_DIFF != ''"
# - name: test nondeterminism
# run: |
# make test-sim-nondeterminism
# if: "env.GIT_DIFF != ''"

test-sim-import-export:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -121,27 +121,3 @@ jobs:
run: |
make test-sim-multi-seed-short
if: "env.GIT_DIFF != ''"

test-liveness:
runs-on: ubuntu-latest
needs: Build
timeout-minutes: 10
steps:
- uses: actions/[email protected]
- uses: actions/[email protected]
with:
go-version: 1.17
- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: start localnet
run: |
make clean localnet-start
if: "env.GIT_DIFF != ''"
- name: test liveness
run: |
./contrib/localnet-blocks-test.sh 100 5 50 localhost
if: "env.GIT_DIFF != ''"
56 changes: 40 additions & 16 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,9 @@
name: Build & Test
on:
pull_request:
paths:
- "**.go"
- go.mod
- go.sum
push:
branches:
- main
paths:
- "**.go"
- go.mod
- go.sum

jobs:
cleanup-runs:
Expand All @@ -28,9 +20,13 @@ jobs:
- uses: actions/[email protected]
with:
go-version: 1.17

- uses: actions/[email protected]

- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- uses: actions/[email protected]
with:
path: |
Expand All @@ -39,14 +35,11 @@ jobs:
key: ${{ runner.os }}-go-${{ hashFiles('**/go.mod') }}
restore-keys: |
${{ runner.os }}-go-

- name: build
run: make build

- name: test & coverage report creation
run: |
make test-unit-cover

- name: filter out DONTCOVER
run: |
excludelist="$(find ./ -type f -name '*.go' | xargs grep -l 'DONTCOVER')"
Expand All @@ -70,11 +63,42 @@ jobs:
- uses: actions/[email protected]
with:
go-version: 1.17

- uses: actions/checkout@v2

- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Build Docker Image
run: make docker-build-debug

- name: Test E2E
run: make test-e2e

liveness-test:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: 1.18
- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Install Ignite CLI
run: |
curl https://get.ignite.com/[email protected]! | bash
if: env.GIT_DIFF
- name: Start Local Network via Ignite CLI
run: |
make start-localnet-ci > ignite.out 2>&1 &
if: env.GIT_DIFF
- name: Test Local Network Liveness
run: |
sleep 3m
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why sleep ? and sleep so long ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignite start a blocking process where it runs a chain, so we run the process in the background and pipe both stdout and stderr to a file. ignite takes a bit to startup (builds app, builds proto, etc...), so the 3m is just a safe "buffer" to wait for ignite to actually start the chain.

./contrib/scripts/test_localnet_liveness.sh 100 5 50 localhost
if: env.GIT_DIFF
23 changes: 8 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -221,17 +221,14 @@ format:
### Localnet ###
###############################################################################

build-docker-gaiadnode:
$(MAKE) -C contrib/testnets/local
start-localnet-ci:
@ignite chain serve --reset-once -v -c ./ignite.ci.yml

# Run a 4-node testnet locally
localnet-start: build-linux localnet-stop
@if ! [ -f build/node0/gaiad/config/genesis.json ]; then docker run --rm -v $(CURDIR)/build:/gaiad:Z tendermint/gaiadnode testnet --v 4 -o . --starting-ip-address 192.168.10.2 --keyring-backend=test ; fi
docker-compose up -d
.PHONY: start-localnet-ci

# Stop testnet
localnet-stop:
docker-compose down
###############################################################################
### Docker ###
###############################################################################

test-docker:
@docker build -f contrib/Dockerfile.test -t ${TEST_DOCKER_REPO}:$(shell git rev-parse --short HEAD) .
Expand All @@ -243,9 +240,5 @@ test-docker-push: test-docker
@docker push ${TEST_DOCKER_REPO}:$(shell git rev-parse --abbrev-ref HEAD | sed 's#/#_#g')
@docker push ${TEST_DOCKER_REPO}:latest

.PHONY: all build-linux install format lint \
go-mod-cache draw-deps clean build \
setup-transactions setup-contract-tests-data start-gaia run-lcd-contract-tests contract-tests \
benchmark \
build-docker-gaiadnode localnet-start localnet-stop \
docker-single-node docker-build-debug docker-build-hermes
.PHONY: all build-linux install format lint go-mod-cache draw-deps clean build \
start-gaia contract-tests benchmark docker-build-debug docker-build-hermes
57 changes: 0 additions & 57 deletions contrib/localnet-blocks-test.sh

This file was deleted.

51 changes: 51 additions & 0 deletions contrib/scripts/test_localnet_liveness.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/bin/bash

CNT=0
ITER=$1
SLEEP=$2
NUMBLOCKS=$3
NODEADDR=$4

if [ -z "$1" ]; then
echo "Invalid argument: missing number of iterations"
echo "sh test_localnet_liveness.sh <iterations> <sleep> <num-blocks> <node-address>"
exit 1
fi

if [ -z "$2" ]; then
echo "Invalid argument: missing sleep duration"
echo "sh test_localnet_liveness.sh <iterations> <sleep> <num-blocks> <node-address>"
exit 1
fi

if [ -z "$3" ]; then
echo "Invalid argument: missing number of blocks"
echo "sh test_localnet_liveness.sh <iterations> <sleep> <num-blocks> <node-address>"
exit 1
fi

if [ -z "$4" ]; then
echo "Invalid argument: missing node address"
echo "sh test_localnet_liveness.sh <iterations> <sleep> <num-blocks> <node-address>"
exit 1
fi

docker_containers=($(docker ps -q -f name=umeed --format='{{.Names}}'))

while [ ${CNT} -lt $ITER ]; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is $ITER used for only counting a max looping ? if this way, can we just write while [ ${CNT} -lt $(expr ${NUMBLOCKS} + 50) ]; do then we do not need to pass the arg ITER=$1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've had this script in the SDK for years at this point. We could probably clean it up. I'm indifferent to it as it's pretty simple.

curr_block=$(curl -s $NODEADDR:26657/status | jq -r '.result.sync_info.latest_block_height')

if [ ! -z ${curr_block} ]; then
echo "Current block: ${curr_block}"
fi

if [ ! -z ${curr_block} ] && [ ${curr_block} -gt ${NUMBLOCKS} ]; then
echo "Success: number of blocks reached"
exit 0
fi

sleep $SLEEP
done

echo "Failed: timeout reached"
exit 1
7 changes: 0 additions & 7 deletions contrib/testnets/local/Makefile

This file was deleted.

24 changes: 0 additions & 24 deletions contrib/testnets/local/gaiadnode/Dockerfile

This file was deleted.

33 changes: 0 additions & 33 deletions contrib/testnets/local/gaiadnode/wrapper.sh

This file was deleted.

Loading