[monitor query] move logs to new azlogs module

eng/config.json

@@ -109,6 +109,10 @@
             "Name": "monitor/azingest",
             "CoverageGoal": 0.75
         },
+        {
+            "Name": "monitor/query/azlogs",
+            "CoverageGoal": 0.85
+        },
         {
             "Name": "security/keyvault/azadmin",
             "CoverageGoal": 0.80

sdk/monitor/query/azlogs/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Release History
+
+## 0.1.0 (2024-03-07)
+
+* This is the initial release of the `azlogs` module

sdk/monitor/query/azlogs/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Microsoft Corporation. All rights reserved.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE

sdk/monitor/query/azlogs/README.md

@@ -0,0 +1,115 @@
+# Azure Monitor Query client module for Go
+
+The Azure Monitor Query client module is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s log data platform.
+
+[Logs][logs_overview] collects and organizes log and performance data from monitored resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual machines agents, and usage and performance data from apps can be consolidated into a single [Azure Log Analytics workspace][log_analytics_workspace]. The various data types can be analyzed together using the [Kusto Query Language][kusto_query_language]. See the [Kusto to SQL cheat sheet][kusto_to_sql] for more information.
+
+Source code | Package (pkg.go.dev) | [REST API documentation][monitor_rest_docs] | [Product documentation][monitor_docs] | Samples
+
+## Getting started
+
+### Prerequisites
+
+* Go, version 1.18 or higher - [Install Go](https://go.dev/doc/install)
+* Azure subscription - [Create a free account][azure_sub]
+* To query Logs, you need one of the following things:
+  * An [Azure Log Analytics workspace][log_analytics_workspace_create]
+  * The resource URI of an Azure resource (Storage Account, Key Vault, Cosmos DB, etc.)
+
+### Install the packages
+
+Install the `azlogs` and `azidentity` modules with `go get`:
+
+```bash
+go get github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs
+go get github.com/Azure/azure-sdk-for-go/sdk/azidentity
+```
+
+The [azidentity][azure_identity] module is used for Azure Active Directory authentication during client construction.
+
+### Authentication
+
+An authenticated client object is required to execute a query. The examples demonstrate using [azidentity.NewDefaultAzureCredential][default_cred_ref] to authenticate; however, the client accepts any [azidentity][azure_identity] credential. See the [azidentity][azure_identity] documentation for more information about other credential types.
+
+The clients default to the Azure public cloud. For other cloud configurations, see the [cloud][cloud_documentation] package documentation.
+
+#### Create a client
+
+Example client.
+
+## Key concepts
+
+### Timespan
+
+It's best practice to always query with a timespan (type `TimeInterval`) to prevent excessive queries of the entire logs data set. Log queries use the ISO8601 Time Interval Standard. All time should be represented in UTC. If the timespan is included in both the Kusto query string and `Timespan` field, the timespan is the intersection of the two values.
+
+Use the `NewTimeInterval()` method for easy creation.
+
+### Logs query rate limits and throttling
+
+The Log Analytics service applies throttling when the request rate is too high. Limits, such as the maximum number of rows returned, are also applied on the Kusto queries. For more information, see [Query API][service_limits].
+
+If you're executing a batch logs query, a throttled request will return a `ErrorInfo` object. That object's `code` value will be `ThrottledError`.
+
+### Advanced logs queries
+
+#### Query multiple workspaces
+
+To run the same query against multiple Log Analytics workspaces, add the additional workspace ID strings to the `AdditionalWorkspaces` slice in the `QueryBody` struct. 
+
+When multiple workspaces are included in the query, the logs in the result table are not grouped according to the workspace from which they were retrieved.
+
+#### Increase wait time, include statistics, include render (visualization)
+
+The `LogsQueryOptions` type is used for advanced logs options.
+
+* By default, your query will run for up to three minutes. To increase the default timeout, set `LogsQueryOptions.Wait` to the desired number of seconds. The maximum wait time is 10 minutes (600 seconds).
+
+* To get logs query execution statistics, such as CPU and memory consumption, set `LogsQueryOptions.Statistics` to `true`.
+
+* To get visualization data for logs queries, set `LogsQueryOptions.Visualization` to `true`.
+
+```go
+azlogs.QueryWorkspaceOptions{
+			Options: &azlogs.LogsQueryOptions{
+				Statistics:    to.Ptr(true),
+				Visualization: to.Ptr(true),
+				Wait:          to.Ptr(600),
+			},
+		}
+```
+
+## Examples
+
+Get started with our examples.
+
+## Contributing
+
+This project welcomes contributions and suggestions. Most contributions require you to agree to a [Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights to use your contribution.
+
+When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate
+the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to
+do this once across all repos using our CLA.
+
+This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information, see
+the [Code of Conduct FAQ][coc_faq] or contact [[email protected]][coc_contact] with any additional questions or
+comments.
+
+<!-- LINKS -->
+[azure_identity]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity
+[azure_sub]: https://azure.microsoft.com/free/
+[azure_monitor_overview]: https://learn.microsoft.com/azure/azure-monitor/overview
+[cloud_documentation]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azcore/cloud
+[default_cred_ref]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/azidentity#defaultazurecredential
+[kusto_query_language]: https://learn.microsoft.com/azure/data-explorer/kusto/query/
+[kusto_to_sql]: https://learn.microsoft.com/azure/data-explorer/kusto/query/sqlcheatsheet
+[log_analytics_workspace]: https://learn.microsoft.com/azure/azure-monitor/logs/log-analytics-workspace-overview
+[log_analytics_workspace_create]: https://learn.microsoft.com/azure/azure-monitor/logs/quick-create-workspace
+[logs_overview]: https://learn.microsoft.com/azure/azure-monitor/logs/data-platform-logs
+[monitor_docs]: https://learn.microsoft.com/azure/azure-monitor/
+[monitor_rest_docs]: https://learn.microsoft.com/rest/api/monitor/
+[service_limits]: https://learn.microsoft.com/azure/azure-monitor/service-limits#la-query-api
+[cla]: https://cla.microsoft.com
+[coc]: https://opensource.microsoft.com/codeofconduct/
+[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
+[coc_contact]: mailto:[email protected]

sdk/monitor/query/azlogs/assets.json

@@ -0,0 +1,6 @@
+{
+  "AssetsRepo": "Azure/azure-sdk-assets",
+  "AssetsRepoPrefixPath": "go",
+  "TagPrefix": "go/monitor/query/azlogs",
+  "Tag": "go/monitor/query/azlogs_139fabf226"
+}

sdk/monitor/query/azlogs/autorest.md

@@ -0,0 +1,127 @@
+## Go
+
+``` yaml
+title: Logs Query Client
+clear-output-folder: false
+go: true
+input-file: https://github.com/Azure/azure-rest-api-specs/blob/0373f0edc4414fd402603fac51d0df93f1f70507/specification/operationalinsights/data-plane/Microsoft.OperationalInsights/stable/2022-10-27/OperationalInsights.json
+license-header: MICROSOFT_MIT_NO_VERSION
+module: github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azlogs
+openapi-type: "data-plane"
+output-folder: ../azlogs
+security: "AADToken"
+use: "@autorest/[email protected]"
+inject-spans: true
+version: "^3.0.0"
+
+directive:
+  # delete extra endpoints
+  - from: swagger-document
+    where: $["paths"]
+    transform: >
+        delete $["/workspaces/{workspaceId}/metadata"];
+  - from: swagger-document
+    where: $["x-ms-paths"]
+    transform: >
+        delete $["/{resourceId}/query?disambiguation_dummy"];
+  - from: swagger-document
+    where: $["paths"]
+    transform: >
+        delete $["/$batch"];
+
+  # delete extra operations
+  - remove-operation: Query_Get
+  - remove-operation: Query_ResourceGet
+  - remove-operation: Query_Batch
+
+  # delete metadata and batch models
+  - remove-model: metadataResults
+  - remove-model: metadataCategory
+  - remove-model: metadataSolution
+  - remove-model: metadataResourceType
+  - remove-model: metadataTable
+  - remove-model: metadataFunction
+  - remove-model: metadataQuery
+  - remove-model: metadataApplication
+  - remove-model: metadataWorkspace
+  - remove-model: metadataResource
+  - remove-model: metadataPermissions
+  - remove-model: batchRequest
+  - remove-model: batchQueryRequest
+  - remove-model: batchResponse
+  - remove-model: batchQueryResponse
+  - remove-model: batchQueryResults
+
+ # rename log operations to generate into a separate logs client
+  - rename-operation:
+      from: Query_Execute
+      to: Logs_QueryWorkspace
+  - rename-operation:
+      from: Query_ResourceExecute
+      to: Logs_QueryResource
+
+  # rename Body.Workspaces to Body.AdditionalWorkspaces
+  - from: swagger-document
+    where: $.definitions.queryBody.properties.workspaces
+    transform: $["x-ms-client-name"] = "AdditionalWorkspaces"
+
+  # rename Render to Visualization
+  - from: swagger-document
+    where: $.definitions.queryResults.properties.render
+    transform: $["x-ms-client-name"] = "Visualization"
+
+  # rename Prefer to Options
+  - from: swagger-document
+    where: $.parameters.PreferHeaderParameter
+    transform: $["x-ms-client-name"] = "Options"
+  - from: options.go
+    where: $
+    transform: return $.replace(/Options \*string/g, "Options *LogsQueryOptions");
+  - from: client.go
+    where: $
+    transform: return $.replace(/\*options\.Options/g, "options.Options.preferHeader()");
+
+  # add descriptions for models and constants that don't have them
+  - from: constants.go
+    where: $
+    transform: return $.replace(/type ResultType string/, "//ResultType - Reduces the set of data collected. The syntax allowed depends on the operation. See the operation's description for details.\ntype ResultType string");
+
+  # delete unused error models
+  - from: models.go
+    where: $
+    transform: return $.replace(/(?:\/\/.*\s)+type (?:ErrorResponse|ErrorResponseAutoGenerated|ErrorInfo|ErrorDetail).+\{(?:\s.+\s)+\}\s/g, "");
+  - from: models_serde.go
+    where: $
+    transform: return $.replace(/(?:\/\/.*\s)+func \(\w \*?(?:ErrorResponse|ErrorResponseAutoGenerated|ErrorInfo|ErrorDetail)\).*\{\s(?:.+\s)+\}\s/g, "");
+
+  # add host as field in client struct
+  - from: client.go
+    where: $
+    transform: return $.replace(/host/g, "client.host");
+  - from: client.go
+    where: $
+    transform: return $.replace(/internal \*azcore.Client/g, "host string\n internal *azcore.Client");
+  - from: constants.go
+    where: $
+    transform: return $.replace(/const host = "(.*?)"/, "");
+
+  # change Table.Rows from type [][]byte to type []Row
+  - from: models.go
+    where: $
+    transform: return $.replace(/Rows \[\]\[\]\[\]byte/g, "Rows []Row");
+
+  # change type of timespan from *string to *TimeInterval
+  - from: 
+        - models.go
+        - options.go
+    where: $
+    transform: return $.replace(/Timespan \*string/g, "Timespan *TimeInterval");
+
+  # delete client name prefix from method options and response types
+  - from:
+      - client.go
+      - options.go
+      - response_types.go
+    where: $
+    transform: return $.replace(/Client(\w+)((?:Options|Response))/g, "$1$2");
+```

sdk/monitor/query/azlogs/build.go

@@ -0,0 +1,10 @@
+//go:build go1.18
+// +build go1.18
+
+//go:generate autorest ./autorest.md --rawjson-as-bytes
+//go:generate gofmt -w .
+
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License. See License.txt in the project root for license information.
+
+package azlogs

sdk/monitor/query/azlogs/ci.yml

@@ -0,0 +1,36 @@
+# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.
+trigger:
+  branches:
+    include:
+    - main
+    - feature/*
+    - hotfix/*
+    - release/*
+  paths:
+    include:
+    - sdk/monitor/query/azlogs
+
+pr:
+  branches:
+    include:
+    - main
+    - feature/*
+    - hotfix/*
+    - release/*
+  paths:
+    include:
+    - sdk/monitor/query/azlogs
+
+
+stages:
+- template: /eng/pipelines/templates/jobs/archetype-sdk-client.yml
+  parameters:
+    ServiceDirectory: 'monitor/query/azlogs'
+    RunLiveTests: true
+    UsePipelineProxy: false
+    SupportedClouds: 'Public,UsGov,China'
+    EnvVars:
+      AZURE_CLIENT_ID: $(AZLOGS_CLIENT_ID)
+      AZURE_TENANT_ID: $(AZLOGS_TENANT_ID)
+      AZURE_CLIENT_SECRET: $(AZLOGS_CLIENT_SECRET)
+      AZURE_SUBSCRIPTION_ID: $(AZLOGS_SUBSCRIPTION_ID)