Skip to content

Commit

Permalink
fix: Bump inotify limits on tester (#11217)
Browse files Browse the repository at this point in the history
Assuming the current default for inotify max user watches on the tester
machine is 8192, this PR bumps it to 64k. We were getting errors when
trying to tail k8s logs caused by hitting inotify limits, for example:

```
spartan-aztec-network-boot-node-0 wait-for-ethereum export ETHEREUM_HOST=http://spartan-aztec-network-ethereum.smoke:8545
spartan-aztec-network-boot-node-0 wait-for-ethereum export BOOT_NODE_HOST=http://spartan-aztec-network-boot-node.smoke:8080
spartan-aztec-network-boot-node-0 wait-for-ethereum export PROVER_NODE_HOST=http://spartan-aztec-network-prover-node.smoke:8080
spartan-aztec-network-boot-node-0 wait-for-ethereum export PROVER_BROKER_HOST=http://spartan-aztec-network-prover-broker.smoke:8084
spartan-aztec-network-boot-node-0 wait-for-ethereum Awaiting ethereum node at http://spartan-aztec-network-ethereum.smoke:8545
spartan-aztec-network-boot-node-0 wait-for-ethereum Waiting for Ethereum node http://spartan-aztec-network-ethereum.smoke:8545...
spartan-aztec-network-boot-node-0 wait-for-ethereum to create fsnotify watcher: too many open files
```
  • Loading branch information
spalladino authored Jan 14, 2025
1 parent 2d88497 commit 60bdf1d
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 1 deletion.
1 change: 1 addition & 0 deletions .github/ensure-tester/run
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ ttl=$1
scripts/run_on_tester "
set -eu;
sudo shutdown -P $ttl;
sudo sysctl fs.inotify.max_user_watches=65536
function clone {
if ! [ -d ~/run-$RUN_ID ]; then
mkdir -p ~/run-$RUN_ID;
Expand Down
1 change: 0 additions & 1 deletion yarn-project/end-to-end/scripts/network_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ fi

STERN_PID=""
function copy_stern_to_log() {
ulimit -n 4096
stern spartan -n $NAMESPACE >$SCRIPT_DIR/network-test.log &
echo "disabled until less resource intensive solution than stern implemented" >$SCRIPT_DIR/network-test.log &
STERN_PID=$!
Expand Down

0 comments on commit 60bdf1d

Please sign in to comment.