Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sqlmigrations: fatal error on CTRL+C during startup #19579

Closed
tbg opened this issue Oct 26, 2017 · 1 comment
Closed

sqlmigrations: fatal error on CTRL+C during startup #19579

tbg opened this issue Oct 26, 2017 · 1 comment
Assignees
@tbg

Comments

@tbg
Copy link
Member

tbg commented Oct 26, 2017

Run rm -rf cockroach-data/; ./cockroach start --insecure --logtostderr a few times, CTRL+C within the first hundred milliseconds to a second

F171026 20:29:57.862364 11 server/server.go:1169  [n1] failed to persist record of completing migration "populate initial version cluster setting table entry": node unavailable; try another peer
goroutine 11 [running]:
github.com/cockroachdb/cockroach/pkg/util/log.getStacks(0x6c8c201, 0x14f138cbbdf616d7, 0x666ceca, 0x10)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:833 +0xcf
github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).outputLogEntry(0x6c8c2c0, 0xc400000004, 0x666ceca, 0x10, 0x491, 0xc4200a4990, 0x90)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:738 +0x5fc
github.com/cockroachdb/cockroach/pkg/util/log.addStructured(0x6766860, 0xc4206d5710, 0x4, 0x2, 0x0, 0x0, 0xc420f95330, 0x1, 0x1)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/util/log/structured.go:153 +0x2fc
github.com/cockroachdb/cockroach/pkg/util/log.logDepth(0x6766860, 0xc4206d5710, 0x1, 0x4, 0x0, 0x0, 0xc420f95330, 0x1, 0x1)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:56 +0x8c
github.com/cockroachdb/cockroach/pkg/util/log.Fatal(0x6766860, 0xc4206d5710, 0xc420f95330, 0x1, 0x1)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/util/log/log.go:159 +0x76
github.com/cockroachdb/cockroach/pkg/server.(*Server).Start(0xc4201fcf00, 0x6766860, 0xc4206d5710, 0x0, 0x0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1169 +0x3243
github.com/cockroachdb/cockroach/pkg/cli.runStart.func1.2(0x6766860, 0xc42074efc0, 0xc4207ca028, 0xc4207ac5a0, 0xc42026e1a0, 0x27d950ae, 0xed1843cc5, 0x0, 0x0, 0x0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:574 +0x1b6
github.com/cockroachdb/cockroach/pkg/cli.runStart.func1(0xc4207ca028, 0x6766860, 0xc42074efc0, 0x6775f00, 0xc42078ac30, 0xc4207ac5a0, 0xc42026e1a0, 0x27d950ae, 0xed1843cc5, 0x0, ...)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:642 +0x11c
created by github.com/cockroachdb/cockroach/pkg/cli.runStart
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:541 +0x467

goroutine 1 [select]:
github.com/cockroachdb/cockroach/pkg/cli.runStart(0x6c701e0, 0xc4202af460, 0x0, 0x2, 0x0, 0x0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/start.go:722 +0xa76
github.com/cockroachdb/cockroach/pkg/cli.MaybeDecorateGRPCError.func1(0x6c701e0, 0xc4202af460, 0x0, 0x2, 0xc4206b3ca0, 0x51a5830)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/error.go:35 +0x57
github.com/cockroachdb/cockroach/pkg/cli.MaybeShoutError.func1(0x6c701e0, 0xc4202af460, 0x0, 0x2, 0x0, 0x0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/error.go:66 +0x51
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).execute(0x6c701e0, 0xc4202af420, 0x2, 0x2, 0x6c701e0, 0xc4202af420)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:698 +0x47a
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x6c72820, 0x402e7b9, 0xc420260080, 0xc4206b3ef0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:783 +0x30e
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).Execute(0x6c72820, 0xc4206b3ee8, 0x0)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:736 +0x2b
github.com/cockroachdb/cockroach/pkg/cli.Run(0xc420010150, 0x3, 0x3, 0xc420014088, 0xc420028c00)
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:159 +0x6d
github.com/cockroachdb/cockroach/pkg/cli.Main()
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:52 +0x16c
main.main()
	/Users/tschottdorf/go/src/github.com/cockroachdb/cockroach/main.go:26 +0x20
@tbg tbg assigned vivekmenezes and tbg and unassigned vivekmenezes Oct 26, 2017
@tbg
Copy link
Member Author

tbg commented Oct 26, 2017

I'll fix this and post it with the repro acceptance test.

tbg added a commit to tbg/cockroach that referenced this issue Oct 26, 2017
@tbg
server: avoid Fatal when draining early
1d38bf4 
Fixes cockroachdb#19579.
tbg added a commit to tbg/cockroach that referenced this issue Oct 26, 2017
@tbg
acceptance: add TestRapidRestarts
ad4929c 
See cockroachdb#19559, cockroachdb#19579, cockroachdb#19571 for issues touched by this test (all of which
it reproduced before the preceding commits fixed them).
@tbg tbg mentioned this issue Oct 26, 2017
Merged
tbg added a commit to tbg/cockroach that referenced this issue Oct 29, 2017
@tbg
server: avoid Fatal when draining early
09fd4aa 
Fixes cockroachdb#19579.
tbg added a commit to tbg/cockroach that referenced this issue Oct 29, 2017
@tbg
acceptance: add TestRapidRestarts
5d8c911 
See cockroachdb#19559, cockroachdb#19579, cockroachdb#19571 for issues touched by this test (all of which
it reproduced before the preceding commits fixed them).

The changes here (outside of the test) ensure that a single-node instance starts
as quickly as possible, without waiting for ports to be ready or for replication
to have occurred. This was done to remove any additional delay in
`TestRapidRestarts` that could otherwise obscur bugs (for example, consider that
going through `./cockroach quit` was not able to tickle any of the bugs). #
Please enter the commit message for your changes. Lines starting # with '#' will
be ignored, and an empty message aborts the commit.

I was also tempted to "bump" the version upgrade tests to use 1.1 instead of 1.0
since that would have removed a bunch of cruft (since 1.0 doesn't write its
listener files), but I decided against that. As a result, some of the new
accessors look a little unclean, but they should straighten themselves out once
we only rely on listener files.
tbg added a commit to tbg/cockroach that referenced this issue Oct 30, 2017
@tbg
acceptance: add TestRapidRestarts
7a0d498 
See cockroachdb#19559, cockroachdb#19579, cockroachdb#19571 for issues touched by this test (all of which
it reproduced before the preceding commits fixed them).

The changes here (outside of the test) ensure that a single-node instance starts
as quickly as possible, without waiting for ports to be ready or for replication
to have occurred. This was done to remove any additional delay in
`TestRapidRestarts` that could otherwise obscur bugs (for example, consider that
going through `./cockroach quit` was not able to tickle any of the bugs). #
Please enter the commit message for your changes. Lines starting # with '#' will
be ignored, and an empty message aborts the commit.

I was also tempted to "bump" the version upgrade tests to use 1.1 instead of 1.0
since that would have removed a bunch of cruft (since 1.0 doesn't write its
listener files), but I decided against that. As a result, some of the new
accessors look a little unclean, but they should straighten themselves out once
we only rely on listener files.
tbg added a commit to tbg/cockroach that referenced this issue Oct 30, 2017
@tbg
acceptance: add TestRapidRestarts
4738e5e 
See cockroachdb#19559, cockroachdb#19579, cockroachdb#19571 for issues touched by this test (all of which
it reproduced before the preceding commits fixed them).

The changes here (outside of the test) ensure that a single-node instance starts
as quickly as possible, without waiting for ports to be ready or for replication
to have occurred. This was done to remove any additional delay in
`TestRapidRestarts` that could otherwise obscur bugs (for example, consider that
going through `./cockroach quit` was not able to tickle any of the bugs). #
Please enter the commit message for your changes. Lines starting # with '#' will
be ignored, and an empty message aborts the commit.

I was also tempted to "bump" the version upgrade tests to use 1.1 instead of 1.0
since that would have removed a bunch of cruft (since 1.0 doesn't write its
listener files), but I decided against that. As a result, some of the new
accessors look a little unclean, but they should straighten themselves out once
we only rely on listener files.
@tbg tbg closed this as completed in edfe09a Oct 30, 2017
@tbg tbg mentioned this issue Sep 27, 2018
Closed
@nvanbenschoten nvanbenschoten mentioned this issue Mar 31, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tbg tbg

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tbg @vivekmenezes