Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make analytics.Client more thread-safe #283

Merged
3 commits merged into from
Feb 23, 2023
Merged

make analytics.Client more thread-safe #283

3 commits merged into from
Feb 23, 2023

Conversation

ghost
Copy link

@ghost ghost commented Feb 23, 2023

It's still not fully thread-safe, since we depend on being able to
append properties to it in klothomain; but for individual sends, it is.

We do this by separating out the client from the payload. Each time we
send a new message, we create a new payload, populate it with default
stuff from the client, modify that paylod (which is thread-local) and
send it.

This PR has two commits (at least for rev1):

  • ff5d15b adds some simple unit tests for the analytics client, but doesn't affect prod at all
  • 9ef7fd0 is the main part of this PR
    • note in particular the change to client_test.go line 91: when we send analytics directly (not via the logger hook), we now send "status": "info" where we didn't before

Followup to #281.

Standard checks

  • Unit tests: added
  • Docs: n/a
  • Backwards compatibility: adds a status field to analytics; see above

Yuval Shavit added 2 commits February 23, 2023 12:09
It's still not fully thread-safe, since we depend on being able to
append properties to it in klothomain; but for individual sends, it is.

We do this by separating out the client from the payload. Each time we
send a new message, we create a new payload, populate it with default
stuff from the client, modify that paylod (which is thread-local) and
send it.
@ghost ghost requested a review from gordon-klotho February 23, 2023 17:57
@ghost
Copy link
Author

ghost commented Feb 23, 2023

I found myself getting a bit lost as I was writing this (much of it yesterday, when I was a bit foggy), so please do take a look to see if I did something wonky in terms of an awkward internal API or struct usage. It's probably due to piecemeal evolving code, rather than a principled stance on code structure.

Copy link
Contributor

@gordon-klotho gordon-klotho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking

logLevel = Panic
}

p := fl.client.createPayload(logLevel, entry.Level.CapitalString())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the event be the same (like "log") since the level is already in the properties? Seems odd to have this duplicated is all

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I prefer this a bit, just because it makes it a bit easier as you're scanning the all-events dashboard to visually pick out warnings vs logs.


return nil
if err != nil {
zap.L().Debug(fmt.Sprintf("Failed to send metrics info. %v", err))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could cause infinite loop if the server is down. Also, the previous version returning the error to let the caller decide what to do makes the most sense IMO.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The infinite loop was there before, too :-\ I noticed it but didn't feel like thinking about how to clean it up, heh.

Regarding returning the error: in the previous version, there was only one caller,(*Client).track:

func (t *Client) track(event string) {
	t.Event = event
	err := SendTrackingToServer(t)

	if err != nil {
		zap.L().Debug(fmt.Sprintf("Failed to send metrics info. %v", err))
	}
}

So:

  • SendTrackingToServer returns a possible error
  • Track is the only invoker of SendTrackingToServer, and the only real value it provides is to turn that error into a log line

It felt like inlining those made sense. That said, maybe it would be clearer if I (a) un-exported SendTrackingToServer and (b) moved it into client.go?

- un-export `send`
- have `createPayload` and `send` use the Payload directly (not pointer)
- create `AppendProperty`
- tweak Sprintf call
@github-actions
Copy link

Package Line Rate Health
github.com/klothoplatform/klotho/pkg/analytics 46%
github.com/klothoplatform/klotho/pkg/annotation 23%
github.com/klothoplatform/klotho/pkg/cli 4%
github.com/klothoplatform/klotho/pkg/core 21%
github.com/klothoplatform/klotho/pkg/env_var 82%
github.com/klothoplatform/klotho/pkg/exec_unit 54%
github.com/klothoplatform/klotho/pkg/infra/kubernetes 59%
github.com/klothoplatform/klotho/pkg/infra/kubernetes/helm 39%
github.com/klothoplatform/klotho/pkg/input 72%
github.com/klothoplatform/klotho/pkg/lang 38%
github.com/klothoplatform/klotho/pkg/lang/dockerfile 0%
github.com/klothoplatform/klotho/pkg/lang/golang 36%
github.com/klothoplatform/klotho/pkg/lang/javascript 48%
github.com/klothoplatform/klotho/pkg/lang/python 63%
github.com/klothoplatform/klotho/pkg/lang/yaml 0%
github.com/klothoplatform/klotho/pkg/logging 23%
github.com/klothoplatform/klotho/pkg/multierr 95%
github.com/klothoplatform/klotho/pkg/provider/aws 49%
github.com/klothoplatform/klotho/pkg/provider/aws/resources 70%
github.com/klothoplatform/klotho/pkg/runtime 78%
github.com/klothoplatform/klotho/pkg/static_unit 33%
github.com/klothoplatform/klotho/pkg/updater 30%
github.com/klothoplatform/klotho/pkg/validation 74%
github.com/klothoplatform/klotho/pkg/yaml_util 79%
Summary 44% (4352 / 9944)

@ghost ghost merged commit 5972792 into main Feb 23, 2023
@ghost ghost deleted the threadsafe-analytics branch February 23, 2023 20:43
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant