forked from apache/hop
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
google sheets and analytics transforms version upgrade, initial doc v…
…ersions apache#3034 apache#2657
- Loading branch information
Showing
19 changed files
with
316 additions
and
92 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
...op-user-manual/modules/ROOT/assets/images/transforms/icons/google-analytics.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
84 changes: 84 additions & 0 deletions
84
docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/google-analytics.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| //// | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, | ||
| software distributed under the License is distributed on an | ||
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations | ||
| under the License. | ||
| //// | ||
| :documentationPath: /pipeline/transforms/ | ||
| :language: en_US | ||
| :description: The Google Analytics Input transform reads data from a Google Analytics 4 account. | ||
|
|
||
| = image:transforms/icons/google-analytics.svg[Google Analytics Input transform Icon, role="image-doc-icon"] Google Analytics Input | ||
|
|
||
| [%noheader,cols="3a,1a", role="table-no-borders" ] | ||
| |=== | ||
| | | ||
| == Description | ||
|
|
||
| The Google Analytics Input transform reads data from a Google Analytics 4 account, using the Google Analytics Data API. | ||
|
|
||
| The https://ga-dev-tools.google/ga4/query-explorer/[GA4 Query Explorer] provides a dedicated website to developing and testing queries with the Google Analytics API (outside of Apache Hop). | ||
|
|
||
| | | ||
| == Supported Engines | ||
| [%noheader,cols="2,1a",frame=none, role="table-supported-engines"] | ||
| !=== | ||
| !Hop Engine! image:check_mark.svg[Supported, 24] | ||
| !Spark! image:question_mark.svg[Maybe Supported, 24] | ||
| !Flink! image:question_mark.svg[Maybe Supported, 24] | ||
| !Dataflow! image:question_mark.svg[Maybe Supported, 24] | ||
| !=== | ||
| |=== | ||
|
|
||
|
|
||
| == Configuration | ||
|
|
||
| * You need a Google Analytics account with access and sufficient permissions to a GA4 property. | ||
| * You need a Google Cloud project with a service account. The Google Analytics api needs to be enabled for this project. | ||
|
|
||
| Check the Google Cloud docs for more information on how the https://cloud.google.com/iam/docs/service-accounts-create[create a service account^] and to https://developers.google.com/analytics/devguides/reporting/data/v1/quickstart-client-libraries[set up a project^] for use with the Google Analytics Data API. | ||
|
|
||
|
|
||
| == Options | ||
|
|
||
| === Google Analytics Connection Settings | ||
|
|
||
| [%header, width="90%"] | ||
| |=== | ||
| |Option|Description | ||
| |Application name|Enter an application name such as "Apache Hop" or leave the default. | ||
| |OAuth service email|Your Google Developer Service Account's Email Address (e.g. "<random [email protected]>"). | ||
| |Key file|The path to the P12 private key associated with your OAuth Service account. | ||
| |Property Id|the GA4 property to read from | ||
| |=== | ||
|
|
||
| === Query Definition | ||
|
|
||
| [%header, width="90%"] | ||
| |=== | ||
| |Option|Description | ||
| |Start date (YYYY-MM-DD|Specifies the start date associated with the query. The date must be entered in the following format: YYYY-MM-DD. | ||
| |End date (YYYY-MM-DD|Specifies the end date associated with the query. The date must be entered in the following format: YYYY-MM-DD. | ||
| |Dimension|Specifies the dimension fields for which you want to query. The Google Analytics Data API documentation provides you with a list of valid inputs and metrics that can be combined. | ||
| |Metrics|Specifies the metrics fields you want returned. At least one metric must be provided. | ||
| |Sort|Specifies a dimension field on which to sort. | ||
| |=== | ||
|
|
||
| === Fields | ||
|
|
||
| Click **Get Fields** to retrieve a list of possible fields based on the query you defined. | ||
| Click **Preview** to preview data based on the defined query. | ||
|
|
||
| === Limit Size | ||
|
|
||
| Limit the number of rows to retrieve for the specified GA4 property to a number of rows. |
62 changes: 62 additions & 0 deletions
62
...hop-user-manual/modules/ROOT/pages/pipeline/transforms/google-sheets-input.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| //// | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, | ||
| software distributed under the License is distributed on an | ||
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations | ||
| under the License. | ||
| //// | ||
| :documentationPath: /pipeline/transforms/ | ||
| :language: en_US | ||
| :description: The Google Sheets Input transform reads data from a Google Sheets worksheets. | ||
|
|
||
| = image:transforms/icons/google-sheets-input.svg[Google Sheets Input transform Icon, role="image-doc-icon"] Google Sheets Input | ||
|
|
||
| [%noheader,cols="3a,1a", role="table-no-borders" ] | ||
| |=== | ||
| | | ||
| == Description | ||
|
|
||
| The Google Sheets Input transform reads data from a Google Sheets worksheets. | ||
|
|
||
| This transform requires a Google service account (JSON file) and a Google Cloud project with the Sheets API enabled. | ||
|
|
||
| | | ||
| == Supported Engines | ||
| [%noheader,cols="2,1a",frame=none, role="table-supported-engines"] | ||
| !=== | ||
| !Hop Engine! image:check_mark.svg[Supported, 24] | ||
| !Spark! image:question_mark.svg[Maybe Supported, 24] | ||
| !Flink! image:question_mark.svg[Maybe Supported, 24] | ||
| !Dataflow! image:question_mark.svg[Maybe Supported, 24] | ||
| !=== | ||
| |=== | ||
|
|
||
| == Options | ||
|
|
||
| === Service account tab | ||
|
|
||
| Click the `browse` button to pick your google service account client secret json file. | ||
|
|
||
| The `Test` button Lets you test your access to the API. On success, you should see a success message. | ||
|
|
||
| === Spreadsheet tab | ||
|
|
||
| Lets you specify or browse for spreadsheets existing in the service account drive or for the ones that are shared with the service account email. | ||
|
|
||
| === Fields tab | ||
|
|
||
| Lets you select from the fields of the sheet. | ||
|
|
||
| Fields name are always defined in the first line of the google spreadsheet. | ||
|
|
||
| **Get Fields** lets you get fields and guess their types, format, precision, decimal and group delimiter as well as trim type. | ||
|
|
63 changes: 63 additions & 0 deletions
63
...op-user-manual/modules/ROOT/pages/pipeline/transforms/google-sheets-output.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| //// | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, | ||
| software distributed under the License is distributed on an | ||
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations | ||
| under the License. | ||
| //// | ||
| :documentationPath: /pipeline/transforms/ | ||
| :language: en_US | ||
| :description: The Google Sheets Output transform writes data to a Google Sheets worksheets. | ||
|
|
||
| = image:transforms/icons/google-sheets-output.svg[Google Sheets Output transform Icon, role="image-doc-icon"] Google Sheets Output | ||
|
|
||
| [%noheader,cols="3a,1a", role="table-no-borders" ] | ||
| |=== | ||
| | | ||
| == Description | ||
|
|
||
| The Google Sheets Output transform writes data to a Google Sheets worksheets. | ||
|
|
||
| This transform requires a Google service account (JSON file) and a Google Cloud project with the Sheets API enabled. | ||
|
|
||
| | | ||
| == Supported Engines | ||
| [%noheader,cols="2,1a",frame=none, role="table-supported-engines"] | ||
| !=== | ||
| !Hop Engine! image:check_mark.svg[Supported, 24] | ||
| !Spark! image:question_mark.svg[Maybe Supported, 24] | ||
| !Flink! image:question_mark.svg[Maybe Supported, 24] | ||
| !Dataflow! image:question_mark.svg[Maybe Supported, 24] | ||
| !=== | ||
| |=== | ||
|
|
||
| == Options | ||
|
|
||
| === Service account tab | ||
|
|
||
| Click the `browse` button to pick your google service account client secret json file. | ||
|
|
||
| The `Test` button Lets you test your access to the API. On success, you should see a success message. | ||
|
|
||
| === Spreadsheet tab | ||
|
|
||
| [%header, width="90%"] | ||
| |=== | ||
| |Option|Description | ||
| |Spreadsheet key|Lets you specify or browse for spreadsheets existing in the service account drive or for the ones that are shared with the service account email. If you type in a sheet name (that does not exist in the drive) it will attempt to create a sheet it the "create" checkbox is ticked. | ||
| |Worksheet Id|Should be browsed form the selected spreadsheet key. If you want to create a new file, type in any key that will become the name of the worksheet in the created spreadsheet | ||
| |Append to sheet|Appends the lines **without the header** to an existing spreadsheet. This is incompatible with the create option below. | ||
| |Create new sheet if it does not exist|If the checkbox is checked then if the Spreadsheet Key specified in the field Spreadsheet key does not exist it will create a new spreadsheet within the service account drive (note that this account has no UI) | ||
| |Share Full Rights (RW) with|the Share with user email field lets you specify the email of a user who will get full rights on the freshly created file. | ||
| |Domain Wide Permission|Enables to share with your whole domain (if configured in Google Drive). | ||
| |=== | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.