Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix 'Hubspot' -> 'HubSpot' #7683

Merged
merged 4 commits into from
Nov 9, 2021
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"sourceDefinitionId": "36c891d9-4bd9-43ac-bad2-10e12756272c",
"name": "Hubspot",
"name": "HubSpot",
"dockerRepository": "airbyte/source-hubspot",
"dockerImageTag": "0.1.22",
"dockerImageTag": "0.1.23",
"documentationUrl": "https://docs.airbyte.io/integrations/sources/hubspot",
"icon": "hubspot.svg"
}
Original file line number Diff line number Diff line change
Expand Up @@ -233,10 +233,10 @@
dockerImageTag: 0.1.5
documentationUrl: https://docs.airbyte.io/integrations/sources/harvest
sourceType: api
- name: Hubspot
- name: HubSpot
sourceDefinitionId: 36c891d9-4bd9-43ac-bad2-10e12756272c
dockerRepository: airbyte/source-hubspot
dockerImageTag: 0.1.22
dockerImageTag: 0.1.23
documentationUrl: https://docs.airbyte.io/integrations/sources/hubspot
icon: hubspot.svg
sourceType: api
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/builds.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
| Google Directory API | [![source-google-directory](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-google-directory%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-google-directory) |
| Google Workspace Admin | [![source-google-workspace-admin-reports](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-google-workspace-admin-reports%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-google-workspace-admin-reports) |
| Greenhouse | [![source-greenhouse](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-greenhouse%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-greenhouse) |
| Hubspot | [![source-hubspot](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-hubspot%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-hubspot) |
| HubSpot | [![source-hubspot](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-hubspot%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-hubspot) |
| IBM Db2 | [![source-db2](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-db2%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-db2) |
| Instagram | [![source-instagram](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-instagram%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-instagram) |
| Intercom | [![source-intercom](https://img.shields.io/endpoint?url=https%3A%2F%2Fdnsgjos7lj2fu.cloudfront.net%2Ftests%2Fsummary%2Fsource-intercom-singer%2Fbadge.json)](https://dnsgjos7lj2fu.cloudfront.net/tests/summary/source-intercom) |
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-hubspot/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_hubspot ./source_hubspot
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.22
LABEL io.airbyte.version=0.1.23
LABEL io.airbyte.name=airbyte/source-hubspot
4 changes: 2 additions & 2 deletions airbyte-integrations/connectors/source-hubspot/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Hubspot Source
# HubSpot Source

This is the repository for the Hubspot source connector, written in Python.
This is the repository for the HubSpot source connector, written in Python.
For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/hubspot).

## Local development
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-hubspot/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

setup(
name="source_hubspot",
description="Source implementation for Hubspot.",
description="Source implementation for HubSpot.",
author="Airbyte",
author_email="[email protected]",
packages=find_packages(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from airbyte_cdk.sources.streams.http.requests_native_auth import Oauth2Authenticator
from source_hubspot.errors import HubspotAccessDenied, HubspotInvalidAuth, HubspotRateLimited, HubspotTimeout

# The value is obtained experimentally, Hubspot allows the URL length up to ~16300 symbols,
# The value is obtained experimentally, HubSpot allows the URL length up to ~16300 symbols,
# so it was decided to limit the length of the `properties` parameter to 15000 characters.
PROPERTIES_PARAM_MAX_LENGTH = 15000

Expand Down Expand Up @@ -99,7 +99,7 @@ def retry_after_handler(**kwargs):
def sleep_on_ratelimit(_details):
_, exc, _ = sys.exc_info()
if isinstance(exc, HubspotRateLimited):
# Hubspot API does not always return Retry-After value for 429 HTTP error
# HubSpot API does not always return Retry-After value for 429 HTTP error
retry_after = int(exc.response.headers.get("Retry-After", 3))
logger.info(f"Rate limit reached. Sleeping for {retry_after} seconds")
time.sleep(retry_after + 1) # extra second to cover any fractions of second
Expand All @@ -119,7 +119,7 @@ def log_giveup(_details):


class API:
"""Hubspot API interface, authorize, retrieve and post, supports backoff logic"""
"""HubSpot API interface, authorize, retrieve and post, supports backoff logic"""

BASE_URL = "https://api.hubapi.com"
USER_AGENT = "Airbyte"
Expand Down Expand Up @@ -332,8 +332,8 @@ def _read(self, getter: Callable, params: MutableMapping[str, Any] = None) -> It
properties_list = list(self.properties.keys())
if properties_list:
# TODO: Additional processing was added due to the fact that users receive 414 errors while syncing their streams (issues #3977 and #5835).
# We will need to fix this code when the Hubspot developers add the ability to use a special parameter to get all properties for an entity.
# According to Hubspot Community (https://community.hubspot.com/t5/APIs-Integrations/Get-all-contact-properties-without-explicitly-listing-them/m-p/447950)
# We will need to fix this code when the HubSpot developers add the ability to use a special parameter to get all properties for an entity.
# According to HubSpot Community (https://community.hubspot.com/t5/APIs-Integrations/Get-all-contact-properties-without-explicitly-listing-them/m-p/447950)
# and the official documentation, this does not exist at the moment.
stream_records = {}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@


class Client(BaseClient):
"""Hubspot client, provides methods to discover and read streams"""
"""HubSpot client, provides methods to discover and read streams"""

def __init__(self, start_date, credentials, **kwargs):
self._start_date = start_date
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"documentationUrl": "https://docs.airbyte.io/integrations/sources/hubspot",
"connectionSpecification": {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Hubspot Source Spec",
"title": "HubSpot Source Spec",
"type": "object",
"required": ["start_date", "credentials"],
"additionalProperties": false,
Expand All @@ -21,7 +21,7 @@
"oneOf": [
{
"type": "object",
"title": "Authenticate via Hubspot (Oauth)",
"title": "Authenticate via HubSpot (Oauth)",
"required": [
"redirect_uri",
"client_id",
Expand All @@ -42,20 +42,20 @@
},
"client_id": {
"title": "Client ID",
"description": "Hubspot client_id. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this id.",
"description": "HubSpot client_id. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this id.",
"type": "string",
"examples": ["123456789000"]
},
"client_secret": {
"title": "Client Secret",
"description": "Hubspot client_secret. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this secret.",
"description": "HubSpot client_secret. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this secret.",
"type": "string",
"examples": ["secret"],
"airbyte_secret": true
},
"refresh_token": {
"title": "Refresh token",
"description": "Hubspot refresh_token. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help generating the token.",
"description": "HubSpot refresh_token. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help generating the token.",
"type": "string",
"examples": ["refresh_token"],
"airbyte_secret": true
Expand All @@ -78,7 +78,7 @@
},
"api_key": {
"title": "API key",
"description": "Hubspot API Key. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this key.",
"description": "HubSpot API Key. See our <a href=\"https://docs.airbyte.io/integrations/sources/hubspot\">docs</a> if you need help finding this key.",
"type": "string",
"airbyte_secret": true
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ const ConnectorServiceTypeControl: React.FC<{
? [
"200330b2-ea62-4d11-ac6d-cfe3e3f8ab2b", // Snapchat
"2470e835-feaf-4db6-96f3-70fd645acc77", // Salesforce Singer
"36c891d9-4bd9-43ac-bad2-10e12756272c", // Hubspot
"36c891d9-4bd9-43ac-bad2-10e12756272c", // HubSpot
"71607ba1-c0ac-4799-8049-7f4b90dd50f7", // Google Sheets
"9da77001-af33-4bcd-be46-6252bf9342b9", // Shopify
"d8313939-3782-41b0-be29-b3ca20d8dd3a", // Intercom
Expand Down
2 changes: 1 addition & 1 deletion docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
* [Google Workspace Admin Reports](integrations/sources/google-workspace-admin-reports.md)
* [Greenhouse](integrations/sources/greenhouse.md)
* [Harvest](integrations/sources/harvest.md)
* [Hubspot](integrations/sources/hubspot.md)
* [HubSpot](integrations/sources/hubspot.md)
* [Instagram](integrations/sources/instagram.md)
* [Intercom](integrations/sources/intercom.md)
* [Iterable](integrations/sources/iterable.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Airbyte uses a grading system for connectors to help users understand what to ex
| [Google Sheets](sources/google-sheets.md) | Certified |
| [Google Workspace Admin Reports](sources/google-workspace-admin-reports.md) | Certified |
| [Greenhouse](sources/greenhouse.md) | Beta |
| [Hubspot](sources/hubspot.md) | Certified |
| [HubSpot](sources/hubspot.md) | Certified |
| [Instagram](sources/instagram.md) | Certified |
| [Intercom](sources/intercom.md) | Beta |
| [Iterable](sources/iterable.md) | Beta |
Expand Down
13 changes: 7 additions & 6 deletions docs/integrations/sources/hubspot.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Hubspot
# HubSpot

## Features

Expand All @@ -11,7 +11,7 @@

## Troubleshooting

Check out common troubleshooting issues for the Hubspot connector on our Discourse [here](https://discuss.airbyte.io/tags/c/connector/11/source-hubspot).
Check out common troubleshooting issues for the HubSpot connector on our Discourse [here](https://discuss.airbyte.io/tags/c/connector/11/source-hubspot).

## Supported Tables

Expand All @@ -35,25 +35,25 @@ This source is capable of syncing the following tables and their data:
* [Tickets](https://developers.hubspot.com/docs/api/crm/tickets)
* [Workflows](https://legacydocs.hubspot.com/docs/methods/workflows/v3/get_workflows)

**Note**: Hubspot API currently only supports `quotes` endpoint using API Key, using Oauth it is impossible to access this stream (as reported by [community.hubspot.com](https://community.hubspot.com/t5/APIs-Integrations/Help-with-using-Feedback-CRM-API-and-Quotes-CRM-API/m-p/449104/highlight/true#M44411)).
**Note**: HubSpot API currently only supports `quotes` endpoint using API Key, using Oauth it is impossible to access this stream (as reported by [community.hubspot.com](https://community.hubspot.com/t5/APIs-Integrations/Help-with-using-Feedback-CRM-API-and-Quotes-CRM-API/m-p/449104/highlight/true#M44411)).

## Getting Started \(Airbyte Open-Source / Airbyte Cloud\)

#### Requirements

* Hubspot Account
* HubSpot Account
* Api credentials
* If using Oauth, [scopes](https://legacydocs.hubspot.com/docs/methods/oauth2/initiate-oauth-integration#scopes) enabled for the streams you want to sync

{% hint style="info" %}
Hubspot's API will [rate limit](https://developers.hubspot.com/docs/api/usage-details) the amount of records you can sync daily, so make sure that you are on the appropriate plan if you are planning on syncing more than 250,000 records per day.
HubSpot's API will [rate limit](https://developers.hubspot.com/docs/api/usage-details) the amount of records you can sync daily, so make sure that you are on the appropriate plan if you are planning on syncing more than 250,000 records per day.
{% endhint %}

This connector supports only authentication with API Key. To obtain API key for the account go to settings -&gt; integrations \(under the account banner\) -&gt; api key. If you already have an api key you can use that. Otherwise generated a new one. See [docs](https://knowledge.hubspot.com/integrations/how-do-i-get-my-hubspot-api-key) for more details.

## Rate Limiting & Performance

The connector is restricted by normal Hubspot [rate limitations](https://legacydocs.hubspot.com/apps/api_guidelines).
The connector is restricted by normal HubSpot [rate limitations](https://legacydocs.hubspot.com/apps/api_guidelines).

When connector reads the stream using `API Key` that doesn't have neccessary permissions to read particular stream, like `workflows`, which requires to be enabled in order to be processed, the log message returned to the output and sync operation goes on with other streams available.

Expand Down Expand Up @@ -96,6 +96,7 @@ If you are using Oauth, most of the streams require the appropriate [scopes](htt

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.23 | 2021-11-05 | [7683](https://github.com/airbytehq/airbyte/pull/7683) | bugfix 'Hubspot' -> 'HubSpot' |
| 0.1.22 | 2021-11-03 | [7562](https://github.com/airbytehq/airbyte/pull/7562) | Migrate Hubspot source to CDK structure |
| 0.1.21 | 2021-10-27 | [7405](https://github.com/airbytehq/airbyte/pull/7405) | Change of package `import` from `urllib` to `urllib.parse` |
| 0.1.20 | 2021-10-26 | [7393](https://github.com/airbytehq/airbyte/pull/7393) | Hotfix for `split_properties` function, add the length of separator symbol `,`(`%2C` in HTTP format) to the checking of the summary URL length |
Expand Down
6 changes: 3 additions & 3 deletions docs/project-overview/changelog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ We're going over the changes from 0.29.17 and before... and there's a lot of big
* 🐛 Slack source: Sync operations no longer hang indefinitely.
* 🐛 Jira source: Now uses updated time as the cursor field for incremental sync instead of the created time.
* 🐛 Intercom source: Fixed inconsistency between schema and output data.
* 🐛 Hubspot source: Streams with the items property now have their schemas fixed.
* 🐛 Hubspot source: Empty strings are no longer handled as dates, fixing the deals, companies, and contacts streams.
* 🐛 HubSpot source: Streams with the items property now have their schemas fixed.
* 🐛 HubSpot source: Empty strings are no longer handled as dates, fixing the deals, companies, and contacts streams.
* 🐛 Typeform source: Allows for multiple choices in responses now.
* 🐛 Shopify source: The type for the amount field is now fixed in the schema.
* 🐛 Postgres destination: \u0000\(NULL\) value processing is now fixed.
Expand Down Expand Up @@ -83,7 +83,7 @@ New Destination: Keen
* ✨ Shopify source: The `status` property is now in the `Products` stream.
* ✨ Amazon Seller Partner source: Added support for `GET_MERCHANT_LISTINGS_ALL_DATA` and `GET_FBA_INVENTORY_AGED_DATA` stream endpoints.
* ✨ GitHub source: Existing streams now don't minify the user property.
* ✨ Hubspot source: Updated user-defined custom field schema generation.
* ✨ HubSpot source: Updated user-defined custom field schema generation.
* ✨ Zendesk source: Migrated from Singer to the Airbyte CDK.
* ✨ Amazon Seller Partner source: Migrated to the Airbyte CDK.
* 🐛 Shopify source: Fixed the `products` schema to be in accordance with the API.
Expand Down
Loading