Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

queue: Flaky test TestLevelQueue #15776

Closed
silverwind opened this issue May 7, 2021 · 0 comments · Fixed by #15693
Closed

queue: Flaky test TestLevelQueue #15776

silverwind opened this issue May 7, 2021 · 0 comments · Fixed by #15693
Labels
type/testing

Comments

@silverwind
Copy link
Member

Can reproduce this locally somewhat rarely. CI failure example:

$ while true; do go test -count=1 -run TestLevelQueue ./modules/queue; done
ok    code.gitea.io/gitea/modules/queue 0.457s
ok    code.gitea.io/gitea/modules/queue 0.452s
ok    code.gitea.io/gitea/modules/queue 0.445s
2021/05/07 19:46:40 ...les/queue/manager.go:134:Add() [T] Queue Manager registered: queue-1 (QID: 1)
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:96:Run() [D] level:  Starting
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:104:Run() [T] level:  Waiting til closed
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:147:readToChan() [T] level : Task found: &queue.testData{TestString:"A", TestInt:1}
2021/05/07 19:46:40 .../queue/workerpool.go:251:commonRegisterWorkers() [T] WorkerPool: 1 (for queue-1) adding 1 workers with group id: 1
2021/05/07 19:46:40 .../queue/workerpool.go:93:zeroBoost() [W] WorkerPool: 1 (for queue-1) has zero workers - adding 5 temporary workers for 5m0s
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:147:readToChan() [T] level : Task found: &queue.testData{TestString:"B", TestInt:2}
2021/05/07 19:46:40 .../queue/workerpool.go:420:doWork() [T] Handling: 1 data, [0xc00065edf8]
2021/05/07 19:46:40 .../queue/workerpool.go:420:doWork() [T] Handling: 1 data, [0xc00000ed98]
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:106:Run() [T] level:  Waiting til done
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 .../queue/workerpool.go:396:doWork() [T] Worker shutting down
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:109:Run() [T] level:  Waiting til cleaned
2021/05/07 19:46:40 .../queue/workerpool.go:309:CleanUp() [T] WorkerPool: 1 CleanUp
2021/05/07 19:46:40 .../queue/workerpool.go:321:CleanUp() [T] WorkerPool: 1 CleanUp Done
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:174:Terminate() [T] level:  Terminating
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:186:Terminate() [D] level:  Closing with 2 tasks left in queue
2021/05/07 19:46:40 ...ue/queue_bytefifo.go:191:Terminate() [D] level:  Terminated
2021/05/07 19:46:41 ...les/queue/manager.go:134:Add() [T] Queue Manager registered: queue-2 (QID: 2)
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:96:Run() [D] level:  Starting
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:104:Run() [T] level:  Waiting til closed
2021/05/07 19:46:41 .../queue/workerpool.go:251:commonRegisterWorkers() [T] WorkerPool: 2 (for queue-2) adding 1 workers with group id: 1
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:147:readToChan() [T] level : Task found: &queue.testData{TestString:"A", TestInt:1}
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:147:readToChan() [T] level : Task found: &queue.testData{TestString:"B", TestInt:2}
2021/05/07 19:46:41 .../queue/workerpool.go:412:doWork() [T] Handling: 2 data, [0xc000d085d0 0xc000d08708]
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:174:Terminate() [T] level:  Terminating
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:106:Run() [T] level:  Waiting til done
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:174:Terminate() [T] level:  Terminating
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:156:Shutdown() [T] level:  Shutting down
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:164:Shutdown() [D] level:  Shutdown
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:186:Terminate() [D] level:  Closing with 0 tasks left in queue
2021/05/07 19:46:41 ...ue/queue_bytefifo.go:191:Terminate() [D] level:  Terminated
--- FAIL: TestLevelQueue (0.70s)
    queue_disk_test.go:21:
          Error Trace:  queue_disk_test.go:21
                              workerpool.go:421
                              workerpool.go:276
                              asm_amd64.s:1371
          Error:        Should be true
          Test:         TestLevelQueue
    queue_disk_test.go:21:
          Error Trace:  queue_disk_test.go:21
                              workerpool.go:421
                              workerpool.go:276
                              asm_amd64.s:1371
          Error:        Should be true
          Test:         TestLevelQueue
    queue_disk_test.go:73:
          Error Trace:  queue_disk_test.go:73
          Error:        Not equal:
                        expected: "A"
                        actual  : "B"

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1 +1 @@
                        -A
                        +B
          Test:         TestLevelQueue
    queue_disk_test.go:74:
          Error Trace:  queue_disk_test.go:74
          Error:        Not equal:
                        expected: 1
                        actual  : 2
          Test:         TestLevelQueue
    queue_disk_test.go:77:
          Error Trace:  queue_disk_test.go:77
          Error:        Not equal:
                        expected: "B"
                        actual  : "A"

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1 +1 @@
                        -B
                        +A
          Test:         TestLevelQueue
    queue_disk_test.go:78:
          Error Trace:  queue_disk_test.go:78
          Error:        Not equal:
                        expected: 2
                        actual  : 1
          Test:         TestLevelQueue
FAIL
FAIL  code.gitea.io/gitea/modules/queue 0.781s
FAIL
ok    code.gitea.io/gitea/modules/queue 0.444s
ok    code.gitea.io/gitea/modules/queue 0.485s
ok    code.gitea.io/gitea/modules/queue 0.450s
ok    code.gitea.io/gitea/modules/queue 0.449s
ok    code.gitea.io/gitea/modules/queue 0.448s
ok    code.gitea.io/gitea/modules/queue 0.445s
ok    code.gitea.io/gitea/modules/queue 0.445s
ok    code.gitea.io/gitea/modules/queue 0.442s
ok    code.gitea.io/gitea/modules/queue 0.449s
@silverwind silverwind changed the title Flaky test TestLevelQueue queue: Flaky test TestLevelQueue May 7, 2021
zeripath added a commit to zeripath/gitea that referenced this issue May 7, 2021
@zeripath
Fix intermittent test failure in TestLevelQueue
e01342d 
Fix go-gitea#15776

Signed-off-by: Andrew Thornton <[email protected]>
@zeripath zeripath mentioned this issue May 7, 2021
Closed
@silverwind silverwind mentioned this issue May 7, 2021
Merged
@6543 6543 linked a pull request May 7, 2021 that will close this issue
Multiple Queue improvements: LevelDB Wait on empty, shutdown empty shadow level queue, reduce goroutines etc #15693
Merged
@go-gitea go-gitea locked and limited conversation to collaborators Oct 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
type/testing
Projects
None yet
Milestone
No milestone
1 participant
@silverwind