Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

failing TCP tests #1498

Closed
marten-seemann opened this issue May 18, 2022 · 2 comments · Fixed by #1507
Closed

failing TCP tests #1498

marten-seemann opened this issue May 18, 2022 · 2 comments · Fixed by #1507
Assignees
@MarcoPolo
Labels
kind/bug A bug in existing code (including security flaws)

Comments

@marten-seemann
Copy link
Contributor

Some of our TCP transport tests are failing randomly.

Failing run: https://github.com/libp2p/go-libp2p/runs/6488638370?check_suite_focus=true

This is probably easiest to debug on the branch on which we merge the transport test suite into go-libp2p (the failure also occurred before, now we just have all the code in one place): #1496

=== RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress50Conn10Stream50Msg
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:60: stream reset
      stream_suite.go:60: stream reset
      stream_suite.go:60: stream reset
      stream_suite.go:60: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:143: s.Write(buf): read tcp4 127.0.0.1:50539->127.0.0.1:50538: read: connection reset by peer
      stream_suite.go:143: s.Write(buf): read tcp4 127.0.0.1:50539->127.0.0.1:50538: read: connection reset by peer
      stream_suite.go:143: s.Write(buf): read tcp4 127.0.0.1:50539->127.0.0.1:50538: read: connection reset by peer
      stream_suite.go:143: s.Write(buf): read tcp4 127.0.0.1:50539->127.0.0.1:50538: read: connection reset by peer
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:143: s.Write(buf): stream reset
      stream_suite.go:143: s.Write(buf): stream reset
      stream_suite.go:143: s.Write(buf): stream reset
      stream_suite.go:143: s.Write(buf): stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1000Stream10Msg
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg10MB
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamOpenStress
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamReset
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestProtocols#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestBasic#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestCancel#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestPingPong#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream1Msg#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream100Msg#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress50Conn10Stream50Msg#01
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:143: s.Write(buf): write tcp4 127.0.0.1:50776->127.0.0.1:50774: write: no buffer space available
      stream_suite.go:143: s.Write(buf): write tcp4 127.0.0.1:50776->127.0.0.1:50774: write: no buffer space available
      stream_suite.go:143: s.Write(buf): write tcp4 127.0.0.1:50776->127.0.0.1:50774: write: no buffer space available
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:54: read tcp4 127.0.0.1:50722->127.0.0.1:50721: read: connection reset by peer
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:156: io.ReadFull(s, buf2): stream reset
      stream_suite.go:54: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
      stream_suite.go:83: stream reset
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1000Stream10Msg#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg10MB#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamOpenStress#01
  === RUN   TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamReset#01
  --- FAIL: TestTcpTransport (8.01s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestProtocols (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestBasic (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestCancel (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestPingPong (0.01s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream1Msg (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream100Msg (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg (0.31s)
      --- FAIL: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress50Conn10Stream50Msg (0.65s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1000Stream10Msg (0.55s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg10MB (0.27s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamOpenStress (2.66s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamReset (0.05s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestProtocols#01 (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestBasic#01 (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestCancel#01 (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestPingPong#01 (0.01s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream1Msg#01 (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1Stream100Msg#01 (0.00s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg#01 (0.22s)
      --- FAIL: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress50Conn10Stream50Msg#01 (0.58s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn1000Stream10Msg#01 (0.32s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStress1Conn100Stream100Msg10MB#01 (0.25s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamOpenStress#01 (2.06s)
      --- PASS: TestTcpTransport/github.com/libp2p/go-libp2p/p2p/transport/testsuite.SubtestStreamReset#01 (0.05s)
@marten-seemann marten-seemann added the kind/bug A bug in existing code (including security flaws) label May 18, 2022
@MarcoPolo
Copy link
Collaborator

Have you seen this outside of macOS? I think I've only seen it within macOS

@marten-seemann
Copy link
Contributor Author

Not 100%, but probably I haven't.
I think this is very likely due to

stream_suite.go:143: s.Write(buf): write tcp4 127.0.0.1:50776->127.0.0.1:50774: write: no buffer space available

We could either go and increase TCP buffer sizes, or tame the test. There's probably little added value testing 50 connections as compared to test 5 or 10.

@MarcoPolo MarcoPolo mentioned this issue May 18, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MarcoPolo MarcoPolo

Labels
kind/bug A bug in existing code (including security flaws)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tcp: limit the number of connections in tcp suite test on non-linux hosts libp2p/go-libp2p
2 participants
@MarcoPolo @marten-seemann