Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

synchronized Depth call #11

Merged
merged 3 commits into from
Jan 16, 2020
Merged

synchronized Depth call #11

merged 3 commits into from
Jan 16, 2020

Conversation

ploxiln
Copy link
Member

@ploxiln ploxiln commented Jan 13, 2020

the Depth() method is more "eventually consistent", rather than always correct at the instant it returns, as pointed out by #10 - here is my proposed alternate-style fix

@ploxiln
Copy link
Member Author

ploxiln commented Jan 16, 2020

It looks like nsqd only explicitly queries backend depth when preparing stats:

[pierce@plo-mbp15 nsq]$ ag --context=1 --ignore='*_test.go' '\.Depth\('
nsqd/channel.go
256-func (c *Channel) Depth() int64 {
257: return int64(len(c.memoryMsgChan)) + c.backend.Depth()
258-}

nsqd/topic.go
238-func (t *Topic) Depth() int64 {
239: return int64(len(t.memoryMsgChan)) + t.backend.Depth()
240-}

nsqd/stats.go
26-          Channels:     channels,
27:          Depth:        t.Depth(),
28:          BackendDepth: t.backend.Depth(),
29-          MessageCount: atomic.LoadUint64(&t.messageCount),
--
62-          ChannelName:   c.name,
63:          Depth:         c.Depth(),
64:          BackendDepth:  c.backend.Depth(),
65-          InFlightCount: inflight,

So these Depth functions are only called by GetStats() -> NewChannelStats() / NewTopicStats(), while a few places call GetStats():

  • statsd.go:statsdLoop()
  • http.go:doStats()
  • nsqadmin calls the nsqd http stats endpoint

so anyway I think that's not too frequent

@ploxiln
Copy link
Member Author

ploxiln commented Jan 16, 2020

cc @mreiferson @jehiah

Copy link
Member

@jehiah jehiah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 clever approach

.travis.yml Show resolved Hide resolved
ploxiln and others added 3 commits January 16, 2020 13:33
doesn't find anything yet, but upcoming depth mgmt changes
could use some race detector checks
ioLoop() now feeds depth into the depthChan synchronously,
only when Depth() is trying to read depthChan
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants