Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial changes to add telemetry #2774

Merged
merged 26 commits into from
May 27, 2021

Conversation

bhsubra
Copy link
Contributor

@bhsubra bhsubra commented May 21, 2021

Changes include:

  • Basic infrastructure to add telemetry for feature usage
  • Add telemetry for top level declaration snippet insertion
  • Add telemetry for resource and module body snippet insertion

@bhsubra bhsubra force-pushed the dev/bhsubra/AddTelemetryForSnippetsScenario branch from ce2dd5d to dfe4d43 Compare May 21, 2021 21:01
README.md Outdated Show resolved Hide resolved
"command": "bicep.Telemetry",
"arguments": [
{
"EventName": "TopLevelDeclarationSnippetInsertion",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Rather long telemetry event name. Consider shortening?

Is it possible that later snippet insertions would not be top-level or declaration? I.e. SnippetInsertion not good enough?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to add telemetry for resource and module body snippet insertions as well. We could also do something like - SnippetInsertion/TopLevel
SnippetInsertion/ResourceBody
SnippetInsertion/ModuleBody

I think it would show up as vscode-bicep/SnippetInsertion/TopLevel

@MarcusFelling , do you have any suggestions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, you just have to deal with whatever names you end up with in queries.

Comment on lines 88 to 92
Dictionary<string, string> properties = new Dictionary<string, string>()
{
{ "name", resourceSnippet.Prefix }
};
TelemetryEvent telemetryEvent = new(TelemetryConstants.EventNames.TopLevelDeclarationSnippetInsertion, properties);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be worth creating a static method to set the properties, rather than exposing the constructor - that way as we add telemetry events, they're all contained in one place, and enforcing more control over the structure of the dictionary?

Thinking something like:

var telemetryEvent = TelemetryEvent.CreateTopLevelDeclarationSnippetInsertion(name: resourceSnippet.Prefix)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anthony-c-martin, I refactored this class a bit. Let me know what you think:
5078ab9


namespace Bicep.LanguageServer.Handlers
{
public class BicepTelemetryHandler : ExecuteCommandHandler
Copy link
Member

@anthony-c-martin anthony-c-martin May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should be able to extend ExecuteCommandHandlerBase<TelemetryEvent> instead of ExecuteCommandHandler and avoid having to write your own deserialization logic in the Handle method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here:
ffbef4c

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With omnisharp 19, looks like the base handler doesn't take in a ISerializer. So added back the deserialization logic. Let me know if I am missing something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the base class here:
62d6763

Copy link
Member

@anthony-c-martin anthony-c-martin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments, please take a look!

@bhsubra bhsubra force-pushed the dev/bhsubra/AddTelemetryForSnippetsScenario branch from e4dd5cb to 9ab7544 Compare May 27, 2021 02:54
@bhsubra bhsubra force-pushed the dev/bhsubra/AddTelemetryForSnippetsScenario branch from 9ab7544 to 6de1c07 Compare May 27, 2021 05:22
@@ -67,24 +67,6 @@
"newText": "if (${1:condition}) {\n\t$0\n}"
}
},
{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expected. There was a bug in SnippetsProvider. See corresponding change in SnippetsProvider.cs.

// Flow of events:
// 1. workspace/executeCommand request is sent from the client to the server
// 2. The above triggers telemetry/event from server to client
public class BicepTelemetryHandler : ExecuteCommandHandlerBase<BicepTelemetryEvent>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the base class has been renamed:
https://github.com/OmniSharp/csharp-language-server-protocol/blob/1b6788df2600083c28811913a221ccac7b1d72c9/src/Protocol/Features/Workspace/ExecuteCommandFeature.cs#L118-L141

I still think it's worth using this rather than doing it ourselves, as that way you ensure that the exact same serializer is being used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here:
62d6763

Thank you!

@@ -10,8 +10,8 @@

namespace Bicep.Core {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a generated file - please could you revert the changes to it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted all, but one change in ln 1. Not sure why it's complaining. VS IDE does some auto formatting. I tried copying it from a clean machine, still has a diff in ln 1

Comment on lines 47 to 48
extensionData["eventName"].ToString().Should().Be(TelemetryConstants.EventNames.TopLevelDeclarationSnippetInsertion);
extensionData["properties"].ToString().Should().BeEquivalentToIgnoringNewlines(@"{
Copy link
Member

@anthony-c-martin anthony-c-martin May 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] we have JToken assertions - you should be able to use:

extensionData["eventName"].Should().DeepEqual(TelemetryConstants.EventNames.TopLevelDeclarationSnippetInsertion);
extensionData["properties"].Should().DeepEqual(new JObject {
    ["name"] = "res-aks-cluster",
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need this anymore. Cleaned up here:
62d6763

TestContext,
options =>
{
options.OnTelemetryEvent(telemetry => telemetryReceived.SetResult(telemetry));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use the typed handler here to avoid the dictionary serialization that TelemetryEventParams does:

options.OnTelemetryEvent<BicepTelemetryEvent>(telemetry => telemetryReceived.SetResult(telemetry));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here:
62d6763

await client.ResolveCompletion(completionItem);
await client.Workspace.ExecuteCommand(command);

return telemetryReceived;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just return the data directly to avoid repetition:

await IntegrationTestHelper.WithTimeoutAsync(telemetryReceived.Task);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here:
62d6763

Comment on lines 126 to 134
Command? command = completionItem.Command;

command!.Name.Should().Be(TelemetryConstants.CommandName);

JArray? arguments = command!.Arguments;
BicepTelemetryEvent? telemetryEvent = arguments!.First().ToObject<BicepTelemetryEvent>();

telemetryEvent!.EventName.Should().Be(eventName);
telemetryEvent!.Properties!.Should().Equal(properties);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove all of these assertions about the command, as you're validating them all at the very end - ultimately that's what matters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here:
62d6763


public string? EventName { get; set; }

public static BicepTelemetryEvent Create(string eventName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't explain myself well with the suggestion of implementing Create - what I was thinking was implementing a set of Create methods - e.g. something like:

public record BicepTelemetryEvent : TelemetryEventParams
{
    public string? EventName { get; set; }

    public Dictionary<string, string>? Properties { get; set; }

    public static BicepTelemetryEvent CreateTopLevelDeclarationSnippetInsertion(string name)
        => new BicepTelemetryEvent {
            EventName = TelemetryConstants.EventNames.TopLevelDeclarationSnippetInsertion,
            Properties = new() {
                ["name"] = name,
            },
        };

    public static BicepTelemetryEvent CreateResourceBodySnippetInsertion(string name, string type)
        => new BicepTelemetryEvent {
            EventName = TelemetryConstants.EventNames.ResourceBodySnippetInsertion,
            Properties = new() {
                ["name"] = name,
                ["type"] = type,
            },
        };

    public static BicepTelemetryEvent CreateModuleBodySnippetInsertion(string name)
        => new BicepTelemetryEvent {
            EventName = TelemetryConstants.EventNames.ModuleBodySnippetInsertion,
            Properties = new() {
                ["name"] = name,
            },
        };
}

This has the benefit of grouping all the telemetry 'schemas' in one place, and simplifies the telemetry raising code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely cleaner. Thanks! Updated here:
62d6763

@@ -1,4 +1,4 @@
//------------------------------------------------------------------------------
//------------------------------------------------------------------------------
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted all but this change. Not sure why it's complaining here.

Copy link
Member

@anthony-c-martin anthony-c-martin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@bhsubra bhsubra merged commit 3e01d8d into main May 27, 2021
@bhsubra bhsubra deleted the dev/bhsubra/AddTelemetryForSnippetsScenario branch May 27, 2021 20:15
@MarcusFelling MarcusFelling linked an issue Jun 1, 2021 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

VS Code Extension Telemetry
5 participants