Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

acceptance: TestDockerCLI/test_demo_memory_warning.tcl/runMode=docker failed #72036

Closed
cockroach-teamcity opened this issue Oct 27, 2021 · 4 comments
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@cockroach-teamcity
Copy link
Member

acceptance.TestDockerCLI/test_demo_memory_warning.tcl/runMode=docker failed with artifacts on master @ 55514eec981d7c5b3279a512cc57b3753ff743b2:

Slow failing tests:
TestDockerCLI/test_demo_memory_warning.tcl/runMode=docker - 1649.05s

Slow passing tests:
TestDockerJava - 18.41s
TestDockerNodeJS - 13.06s
TestDockerC - 10.93s
TestDockerRuby - 9.16s
TestDockerPHP - 9.14s
TestDockerPython - 8.85s
TestDockerPSQL - 5.89s
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:

  • GOFLAGS=-json

Internal log

tamird marked as alumn{us/a}; resolving to tbg instead

/cc @cockroachdb/sql-experience tbg

This test on roachdash | Improve this report!

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Oct 27, 2021
@rafiss
Copy link
Collaborator

rafiss commented Oct 29, 2021

@jordanlewis @ajstorm I wonder if it's possible this relates to the new multitenant mode? I'll put it on the board just in case

@ajstorm ajstorm closed this as completed Jan 19, 2022
@RaduBerinde
Copy link
Member

We haven't seen this since, closing.

@tbg
Copy link
Member

tbg commented Feb 8, 2023

Different failure mode seen here on #96257

expect version 5.45.4
argv[0] = expect  argv[1] = -d  argv[2] = -f  argv[3] = /mnt/interactive_tests/test_demo_memory_warning.tcl  argv[4] = /cockroach/cockroach  
set argc 1
set argv0 "/mnt/interactive_tests/test_demo_memory_warning.tcl"
set argv "/cockroach/cockroach"
executing commands from command file /mnt/interactive_tests/test_demo_memory_warning.tcl
system(mkdir -p logs) = 2
system(rm -f cockroach_sql_history) = 2
trap: setting up signal 2 ("SIGINT")
trap: setting up signal 15 ("SIGTERM")

.230208 09:21:18.105815639 EXPECT TEST: START TEST: Check demo alerts when there is a memory warning.
system(echo; echo $(date '+.%y%m%d %H:%M:%S.%N') EXPECT TEST: 'START TEST: Check demo alerts when there is a memory warning.' | tee -a logs/expect-cmd.log) = 2
system(find logs -exec chmod a+rw '{}' \; || true) = 2
spawn /cockroach/cockroach demo --no-line-editor --no-example-database --max-sql-memory=10% --nodes=8
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {30}

expect: does "" (spawn_id exp3) match glob pattern "WARNING: HIGH MEMORY USAGE"? no
*
* WARNING: HIGH MEMORY USAGE
* The sum of --max-sql-memory (10 GiB) and --cache (64 MiB) multiplied by the
* number of nodes (8) results in potentially high memory usage on your
* device.
* This server is running at increased risk of memory-related failures.
*

expect: does "*\r\n* WARNING: HIGH MEMORY USAGE\r\n* The sum of --max-sql-memory (10 GiB) and --cache (64 MiB) multiplied by the\r\n* number of nodes (8) results in potentially high memory usage on your\r\n* device.\r\n* This server is running at increased risk of memory-related failures.\r\n*\r\n" (spawn_id exp3) match glob pattern "WARNING: HIGH MEMORY USAGE"? yes
expect: set expect_out(0,string) "WARNING: HIGH MEMORY USAGE"
expect: set expect_out(spawn_id) "exp3"
expect: set expect_out(buffer) "*\r\n* WARNING: HIGH MEMORY USAGE"

expect: does "\r\n* The sum of --max-sql-memory (10 GiB) and --cache (64 MiB) multiplied by the\r\n* number of nodes (8) results in potentially high memory usage on your\r\n* device.\r\n* This server is running at increased risk of memory-related failures.\r\n*\r\n" (spawn_id exp3) match glob pattern "defaultdb>"? no
#
# Welcome to the CockroachDB demo database!
#
# You are connected to a temporary, in-memory CockroachDB cluster of 8 nodes.
#
# You are connected to tenant "demo-tenant", but can connect to the system tenant with
# '\connect' and the SQL url printed via '\demo ls'.
#
# Telemetry disabled by configuration.

expect: does "\r\n* The sum of --max-sql-memory (10 GiB) and --cache (64 MiB) multiplied by the\r\n* number of nodes (8) results in potentially high memory usage on your\r\n* device.\r\n* This server is running at increased risk of memory-related failures.\r\n*\r\n#\r\n# Welcome to the CockroachDB demo database!\r\n#\r\n# You are connected to a temporary, in-memory CockroachDB cluster of 8 nodes.\r\n#\r\n# You are connected to tenant "demo-tenant", but can connect to the system tenant with\r\n# '\connect' and the SQL url printed via '\demo ls'.\r\n#\r\n# Telemetry disabled by configuration.\r\n" (spawn_id exp3) match glob pattern "defaultdb>"? no
expect: timed out

.230208 09:21:48.780169549 EXPECT TEST: TIMEOUT WAITING FOR "defaultdb>"
system(echo; echo $(date '+.%y%m%d %H:%M:%S.%N') EXPECT TEST: 'TIMEOUT WAITING FOR "defaultdb>"' | tee -a logs/expect-cmd.log) = 2
system(find logs -exec chmod a+rw '{}' \; || true) = 2
non-zero exit code: 1

@tbg tbg reopened this Feb 8, 2023
@ajstorm ajstorm added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Mar 7, 2023
@rafiss
Copy link
Collaborator

rafiss commented Jun 30, 2023

This is no longer flaking.

@rafiss rafiss closed this as completed Jun 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Development

No branches or pull requests

5 participants