Coinbase: Update exchange implementation #1480

cranktakular · 2024-02-14T06:07:40Z

PR Description

Updating Coinbase to the Advanced Trade and Sign In With Coinbase APIs, as well as a few tiny fixes of other parts of the codebase found along the way.

Type of change

Please delete options that are not relevant and add an x in [] as item is complete.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
This change requires a documentation update

How has this been tested

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration and
also consider improving test coverage whilst working on a certain feature or package.

go test ./... -race
golangci-lint run

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation and regenerated documentation via the documentation tool
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally and on Github Actions with my changes

The only tests which seem to be failing (exchanges/kucoin, database/models/postgres, config, and common/file) are parts that I haven't substantively changed, and which seem to be failing on master too.

Continual enhance of Coinbase tests The revamp continues Oh jeez the Orderbook part's unfinished don't look Coinbase revamp, Orderbook still unfinished

…otrader into coinbase_api_revamp

V3 done, onto V2 Coinbase revamp nears completion Coinbase revamp nears completion Test commit should fail Coinbase revamp nears completion

gloriousCode · 2025-01-21T23:00:20Z

exchanges/coinbasepro/coinbasepro_test.go

 	sharedtestvalues.SkipTestIfCredentialsUnset(t, c)
 	resp, err = c.GetProductBookV3(context.Background(), testPair, 4, -1, true)
-	assert.NoError(t, err)
+	require.NoError(t, err)


Just as an FYI, not a required change, asserts' have bool responses, so if you don't like the require, you can do

if assert.NoError(errawr) { assert.NotEmpty(kasjhdfaksdjfhasldkfh) }

Ooh, that's nice.

exchanges/coinbasepro/coinbasepro_websocket.go

gloriousCode

Thank you for resolving the orderbook+ticker sync issue!

gloriousCode · 2025-02-03T00:50:26Z

exchanges/coinbasepro/coinbasepro_websocket.go

@@ -3,30 +3,63 @@ package coinbasepro
 import (


I am getting this when running CBP:

[DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro attempt 1 request path: https://api.coinbase.com/v2/accounts/2b6b9d41-33bd-1337-83ac-dbe5b4221337/addresses [DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro request header [Authorization]: [Bearer: roar] [DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro request header [Content-Type]: [application/json] [DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro request header [Cb-Version]: [2024-11-27] [DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro request type: POST [DEBUG] | REQUESTER | 03/02/2025 11:45:07 | CoinbasePro request body: {"name":""} [ERROR] | LOG | 03/02/2025 11:45:08 | CoinbasePro failed to get cryptocurrency deposit address for ETH. Err: CoinbasePro unsuccessful HTTP status code: 401 raw response: {"errors":[{"id":"internal_server_error","message":"An error has occurred. If this problem persists, please visit https://support.coinbase.com for assistance."}]}, authenticated request failed

However, I receive no error running TestGetAllAddresses:

CoinbasePro attempt 1 request path: https://api.coinbase.com/v2/accounts/2b6b9d41-33bd-1337-83ac-dbe5b4221337/addresses CoinbasePro request header [Content-Type]: [application/json] CoinbasePro request header [Cb-Version]: [2024-11-27] CoinbasePro request header [Authorization]: [Bearer: roar] CoinbasePro request type: GET CoinbasePro request body: HTTP status: 200 OK, Code: 200

Given the success in tests using a get versus an error in POST can you look into what func you're using in normal operations?

Though if nothing is wrong when you test, please le me know since it is saying internal_server_error, it may just go away

For extra context the credentials are the same between my config and tests.

[ERROR] | WEBSOCKET | 03/02/2025 11:52:51 | exchange CoinbasePro websocket error - {"type":"error","message":"authentication failure"}

The function which calls that endpoint with the POST method is CreateAddress. Running that myself, I get the error you mention only intermittently.

So intermittently, in fact, that I've been unable to reproduce it and debug it; I wasn't able to catch verbose output of that error.

If it's still coming up for you, a verbose paste of TestCreateAddress' output would be appreciated!

EDIT: However, thinking about this, it would probably be a functionality improvement if I check for existing addresses before generating a new one; I'll update GetDepositAddress' functionality to do that.

EDIT 2: Done.

The test is intermittent, in GCT I receive it as above 100% of the time. You shouldn't be calling CreateAddress on GetDepositAddress anyway, it's a retrieval endpoint. There are two exchanges where that isn't the case though (I would still argue the same thing), so I understand how you could have looked at those, but just retrieve the deposit address and I imagine the error will cease

edit: no fair on editing and committing while I reply 😄 that is a good improvement, thank you

shazbert

Thanks for the changes! Some more nits getting my head around everything again before deep dive into testing. 🚀

currency/forexprovider/base/base_interface.go

engine/depositaddress.go

engine/rpcserver.go

shazbert · 2025-02-11T02:38:21Z

exchanges/coinbasepro/coinbasepro.go

+	errPayMethodNotFound        = errors.New("payment method not found")
+	errUnknownL2DataType        = errors.New("unknown l2update data type")
+	errOrderFailedToCancel      = errors.New("failed to cancel order")
+	errUnrecognisedStatusType   = errors.New("unrecognised status type")


seems like this could be an order package level error

I can't find it there.

Can create a new status error if one is not found

exchanges/coinbasepro/coinbasepro.go

exchanges/coinbasepro/coinbasepro_wrapper.go

shazbert · 2025-02-11T03:21:13Z

exchanges/coinbasepro/coinbasepro_wrapper.go

+			Total:                  accountBalance[i].AvailableBalance.Value,
+			Hold:                   accountBalance[i].Hold.Value,
+			Free:                   accountBalance[i].AvailableBalance.Value - accountBalance[i].Hold.Value,
+			AvailableWithoutBorrow: accountBalance[i].AvailableBalance.Value,


NOTE 4 me test this.

exchanges/coinbasepro/coinbasepro_wrapper.go

exchanges/exchange.go

shazbert

Just some nits on new changes thanks heaps for going over them! Nice stuff.

shazbert · 2025-02-20T21:37:56Z

exchanges/coinbasepro/coinbasepro.go

+// UnmarshalJSON unmarshals the JSON data
+func (o *Orders) UnmarshalJSON(data []byte) error {
+	var alias any
+	var str1, str2 string


these don't need to be allocated if you change Orders Field value to types.Number and the below ParseFloat can be removed and below

shazbert · 2025-02-20T21:41:06Z

exchanges/coinbasepro/coinbasepro.go

+// UnmarshalJSON unmarshals the JSON data
+func (c *Candle) UnmarshalJSON(data []byte) error {
+	var f1, f2, f3, f4, f5, f6 float64
+	temp := [6]any{&f1, &f2, &f3, &f4, &f5, &f6}


point directly to field of c and change Time feild to types.Time

shazbert · 2025-02-20T21:44:16Z

exchanges/coinbasepro/coinbasepro_websocket.go

 		c.aliasStruct.m.RLock()
-		aliases := make(map[currency.Pair]currency.Pairs)
-		maps.Copy(aliases, c.aliasStruct.associatedAliases)
+		aliases := maps.Clone(c.aliasStruct.associatedAliases)
 		c.aliasStruct.m.RUnlock()


can be GetAliases() method with mutex calls in scope.

shazbert · 2025-02-20T21:47:34Z

exchanges/coinbasepro/coinbasepro_wrapper.go

+	c.aliasStruct.m.Lock()
+	defer c.aliasStruct.m.Unlock()
+	if c.aliasStruct.associatedAliases == nil {
+		c.aliasStruct.associatedAliases = make(map[currency.Pair]currency.Pairs)
+	}
+	for k, v := range aliases {
+		c.aliasStruct.associatedAliases[k] = c.aliasStruct.associatedAliases[k].Add(v...)
+	}


can be method Load(...)

shazbert · 2025-02-20T21:49:56Z

exchanges/exchange.go

+	urlMap := maps.Clone(e.defaults)
 	e.mu.RUnlock()
 	return urlMap


Suggested change

urlMap := maps.Clone(e.defaults)

e.mu.RUnlock()

return urlMap

defer e.mu.RUnlock()

return maps.Clone(e.defaults)

shazbert · 2025-02-20T22:06:47Z

exchanges/coinbasepro/coinbasepro_wrapper.go

@@ -1338,3 +1332,10 @@ func FormatAssetOutbound(a asset.Item) string {
 	}
 	return a.Upper()
 }
+
+// LoadAlias returns the aliases for a currency pair, with the original pair included
+func (a *aliasStruct) LoadAlias(p currency.Pair) currency.Pairs {


Good start, all the other references to Alias struct can be methods as suggested above. Can change name of aliasstruct to productAlias or pairAlias and have specific tests associated for the methods.

shazbert · 2025-02-20T22:08:43Z

exchanges/coinbasepro/coinbasepro.go

 	errEndpointPathInvalid      = errors.New("endpoint path invalid, should start with https://")
 	errPairsDisabledOrErrored   = errors.New("pairs are either disabled or errored")
+	errTypeAssert               = errors.New("type assertion failed")


When common.GetTypeAssertError is called it returns common.ErrTypeAssertFailure so this can be removed

shazbert · 2025-02-20T22:09:23Z

exchanges/coinbasepro/coinbasepro.go

+	errPayMethodNotFound        = errors.New("payment method not found")
+	errUnknownL2DataType        = errors.New("unknown l2update data type")
+	errOrderFailedToCancel      = errors.New("failed to cancel order")
+	errUnrecognisedStatusType   = errors.New("unrecognised status type")


Can create a new status error if one is not found

shazbert

Another round of nits and suggestions for you, really nice work on what you have done so far.

shazbert · 2025-02-24T00:56:21Z

exchanges/coinbasepro/coinbasepro.go

-			Volume: resp[x][5],
+// GetAllOrders lists orders, filtered by their status
+func (c *CoinbasePro) GetAllOrders(ctx context.Context, productID, userNativeCurrency, orderType, orderSide, cursor, productType, orderPlacementSource, contractExpiryType, retailPortfolioID string, orderStatus, assetFilters []string, limit int32, startDate, endDate time.Time) (*GetAllOrdersResp, error) {
+	var params Params


This embeded struct type can just be params := make(url.Values) unless needed for methods across this PR.

It's needed for methods

shazbert · 2025-02-24T01:00:29Z

exchanges/coinbasepro/coinbasepro.go

 	}
-	if clientRef != "" {
-		req["client_oid"] = clientRef
+	if amount == 0 {


missing test coverage

shazbert · 2025-02-24T01:02:21Z

exchanges/coinbasepro/coinbasepro.go

+}
+
+// GetV3Time returns the current server time, calling V3 of the API
+func (c *CoinbasePro) GetV3Time(ctx context.Context) (*ServerTimeV3, error) {


missing test coverage

The test was skipping since the endpoint was authenticated a year ago, and I forgot to update the test when it was made unauth ~9 months ago.

shazbert · 2025-02-24T01:16:46Z

exchanges/coinbasepro/coinbasepro.go

+	err := common.StartEndTimeCheck(startDate, endDate)
+	if err != nil {


Suggested change

err := common.StartEndTimeCheck(startDate, endDate)

if err != nil {

if err := common.StartEndTimeCheck(startDate, endDate); err != nil {

shazbert · 2025-02-24T01:19:10Z

exchanges/coinbasepro/coinbasepro.go

-			}
-			break
+// PrepareDateString encodes a set of parameters indicating start & end dates
+func (p *Params) prepareDateString(startDate, endDate time.Time, labelStart, labelEnd string) error {


maybe change name: encodeDateRange or setDateRange also zero strings are not being checked and the method needs test coverage.

shazbert · 2025-02-24T02:18:03Z

exchanges/coinbasepro/coinbasepro_wrapper.go

-
-	err = ticker.ProcessTicker(tickerPrice)
+// FetchTicker returns the ticker for a currency pair
+func (c *CoinbasePro) FetchTicker(ctx context.Context, p currency.Pair, assetType asset.Item) (*ticker.Price, error) {


RM FetchTicker, Orderbook, AccountInfo aas well as tests as they are now base level methods from

shazbert · 2025-02-24T02:20:49Z

exchanges/coinbasepro/coinbasepro_wrapper.go

 func (c *CoinbasePro) ValidateAPICredentials(ctx context.Context, assetType asset.Item) error {
 	_, err := c.UpdateAccountInfo(ctx, assetType)
 	return c.CheckTransientError(err)
 }

 // GetServerTime returns the current exchange server time.
 func (c *CoinbasePro) GetServerTime(ctx context.Context, _ asset.Item) (time.Time, error) {
-	st, err := c.GetCurrentServerTime(ctx)
+	st, err := c.GetV2Time(ctx)


Used V2Time since V3Time used to be authenticated, but I guess I can switch it now.

shazbert · 2025-02-24T02:23:06Z

exchanges/coinbasepro/coinbasepro_wrapper.go

+}
+
+// processFundingData is a helper function for GetAccountFundingHistory and GetWithdrawalsHistory, transforming the data returned by the Coinbase API into a format suitable for the exchange package
+func (c *CoinbasePro) processFundingData(accHistory []DeposWithdrData, cryptoHistory []TransactionData) []exchange.FundingHistory {


missing test coverage

shazbert · 2025-02-24T02:24:03Z

exchanges/coinbasepro/coinbasepro_wrapper.go

+		return granFifteenMin, nil
+	case kline.ThirtyMin:
+		return granThirtyMin, nil
+	case kline.OneHour:


Missing test coverage

shazbert · 2025-02-24T02:40:03Z

exchanges/coinbasepro/coinbasepro_wrapper.go

+}
+
+// XOR's the current time with the amount to cheaply generate an idempotency token where unwanted collisions should be rare
+func generateIdempotency(am float64) string {


If this is required I would update the withdraw.Request type so that this can be defined outside of this package. I wouldn't recommend, I would prefer a uuid instead I think V7 is a performant option. I don't know enough yet about what is going on to guarantee uniqueness for a edge case we can parallel this request and do two independant withdraws of 1337.1337 and 1337.1336 both will have the same unix nano timestamp and might be the same value u.

It's not required, and is only used for this function of withdrawing money.

Could remove it if you want.

cranktakular added 28 commits October 9, 2023 08:42

Slight enhance of Coinbase tests

a006163

Continual enhance of Coinbase tests The revamp continues Oh jeez the Orderbook part's unfinished don't look Coinbase revamp, Orderbook still unfinished

Merge branch 'master' into coinbase_api_revamp

074f367

Coinbase revamp; CreateReport is still WIP

bd9f2dd

More coinbase improvements; onto sandbox testing

25fcd2e

Coinbase revamp continues

5d3fd99

Coinbase revamp continues

3aca36b

Merge branch 'coinbase_api_revamp' of github.com:cranktakular/gocrypt…

f58b76d

…otrader into coinbase_api_revamp

Coinbasepro revamp is ceaseless

eded160

Coinbase revamp, starting on advanced trade API

5a0ba23

Coinbase Advanced Trade Starts in Ernest

88e6713

V3 done, onto V2 Coinbase revamp nears completion Coinbase revamp nears completion Test commit should fail Coinbase revamp nears completion

Coinbase revamp stage wrapper

aa2efef

Merge branch 'master' into coinbase_api_revamp

3560599

Coinbase wrapper coherence continues

4593828

Coinbase wrapper continues writhing

a8cae40

Coinbase wrapper & codebase cleanup

0564e45

Coinbase updates & wrap progress

4f5ccb5

More Coinbase wrapper progress

9a6c7e6

Wrapper is wrapped, kinda

2896b3e

Test & type checking

ffb43d8

Coinbase REST revamp finished

7dd4589

Merge branch 'master' into coinbase_api_revamp

6d7ac5f

Post-merge fix

2881841

WS revamp begins

2166886

WS Main Revamp Done?

35eda67

CB websocket tidying up

6ac86cd

Coinbase WS wrapperupperer

cfe6e11

Coinbase revamp done??

2f770d5

Merge branch 'master' into coinbase_api_revamp

3e54528

thrasher- changed the title ~~Coinbase api revamp~~ Coinbase: Update exchange implementation Feb 14, 2024

thrasher- requested a review from a team February 14, 2024 23:40

cranktakular added 3 commits January 14, 2025 16:51

Merge branch 'master' into coinbase_api_revamp

2350629

Required changes

3447fea

Stop going

4820d65

gloriousCode reviewed Jan 21, 2025

View reviewed changes

Alias attempt

a316c25

gloriousCode removed the review me This pull request is ready for review label Jan 28, 2025

cranktakular added 3 commits January 28, 2025 17:41

Alias fix & test cleanup

2e35113

Merge branch 'master' into coinbase_api_revamp

50a599b

Test fix

5cc49e8

gloriousCode requested changes Feb 3, 2025

View reviewed changes

gloriousCode added the review me This pull request is ready for review label Feb 3, 2025

cranktakular added 5 commits February 5, 2025 13:03

GetDepositAddress logic improvement

0c85038

Status update: Fixed

2de2256

Lint fix

ea87268

Merge branch 'master' into coinbase_api_revamp

944b975

Merge branch 'master' into coinbase_api_revamp

00d42b6

shazbert requested changes Feb 11, 2025

View reviewed changes

gloriousCode removed the review me This pull request is ready for review label Feb 19, 2025

cranktakular added 3 commits February 20, 2025 16:04

Happy birthday to PR 1480

e33a655

Merge branch 'master' into coinbase_api_revamp

5f8f5b5

Cleanups

7d9789e

shazbert requested changes Feb 20, 2025

View reviewed changes

shazbert requested changes Feb 24, 2025

View reviewed changes

shazbert added the rebase/merge of master required label Feb 24, 2025

cranktakular added 4 commits February 27, 2025 15:42

Necessary nit corrections

fb0fe06

Merge branch 'master' into coinbase_api_revamp

7409cad

Fixing sillybug

5de9d7f

Merge branch 'master' into coinbase_api_revamp

d3bdad0

cranktakular removed the rebase/merge of master required label Mar 6, 2025

gloriousCode added the review me This pull request is ready for review label Mar 11, 2025

		err := common.StartEndTimeCheck(startDate, endDate)
		if err != nil {

	err := common.StartEndTimeCheck(startDate, endDate)
	if err != nil {
	if err := common.StartEndTimeCheck(startDate, endDate); err != nil {

Coinbase: Update exchange implementation #1480

Are you sure you want to change the base?

Coinbase: Update exchange implementation #1480

Conversation

cranktakular commented Feb 14, 2024 • edited Loading

PR Description

Type of change

How has this been tested

Checklist

gloriousCode Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gloriousCode left a comment

Choose a reason for hiding this comment

gloriousCode Feb 3, 2025 • edited Loading

Choose a reason for hiding this comment

gloriousCode Feb 3, 2025 • edited Loading

Choose a reason for hiding this comment

cranktakular Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

gloriousCode Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

shazbert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shazbert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shazbert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cranktakular commented Feb 14, 2024 •

edited

Loading

gloriousCode Jan 21, 2025 •

edited

Loading

gloriousCode Feb 3, 2025 •

edited

Loading

gloriousCode Feb 3, 2025 •

edited

Loading

cranktakular Feb 5, 2025 •

edited

Loading

gloriousCode Feb 5, 2025 •

edited

Loading