Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature(websocket): Add websocket client #508

Merged
merged 14 commits into from
Jul 19, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,41 @@ To receive results in ``pandas`` format, use the ``get_dataframe()`` method:
startDate='2017-01-01',
endDate='2018-05-31')

Websocket support::

.. code-block:: python
from tiingo import TiingoWebsocketClient

def cb_fn(msg):

# Example response
# msg = {
# "service":"iex" # An identifier telling you this is IEX data.
# The value returned by this will correspond to the endpoint argument.
#
# # Will always return "A" meaning new price quotes. There are also H type Heartbeat msgs used to keep the connection alive
# "messageType":"A" # A value telling you what kind of data packet this is from our IEX feed.
#
# # see https://api.tiingo.com/documentation/websockets/iex > Response for more info
# "data":[] # an array containing trade information and a timestamp
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on the documentation for "service" and "messageType"!

Can we provide a sample of what one of these objects would look like? 1 or 2 would be plenty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think adding examples would work out for stylistic reasons. anyways the link provided gives a lot of in-depth details

#
# }

print(msg)

subscribe = {
'eventName':'subscribe',
'authorization':'API_KEY_GOES_HERE',
#see https://api.tiingo.com/documentation/websockets/iex > Request for more info
'eventData': {
'thresholdLevel':5
}
}
# notice how the object isn't needed after using it
# any logic should be implemented in the callback function
TiingoWebsocketClient(subscribe,endpoint="iex",on_msg_cb=cb_fn)
while True:pass


You can specify any of the end of day frequencies (daily, weekly, monthly, and annually) or any intraday frequency for both the ``get_ticker_price`` and ``get_dataframe`` methods. Weekly frequencies resample to the end of day on Friday, monthly frequencies resample to the last day of the month, and annually frequencies resample to the end of day on 12-31 of each year. The intraday frequencies are specified using an integer followed by "Min" or "Hour", for example "30Min" or "1Hour".

Expand Down
35 changes: 35 additions & 0 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,41 @@ Now you can use ``TiingoClient`` to make your API calls. (Other parameters are a
startDate='2017-01-01',
endDate='2017-08-31')

Websocket support::

.. code-block:: python
from tiingo import TiingoWebsocketClient

def cb_fn(msg):

# Example response
# msg = {
# "service":"iex" # An identifier telling you this is IEX data.
# The value returned by this will correspond to the endpoint argument.
#
# # Will always return "A" meaning new price quotes. There are also H type Heartbeat msgs used to keep the connection alive
# "messageType":"A" # A value telling you what kind of data packet this is from our IEX feed.
#
# # see https://api.tiingo.com/documentation/websockets/iex > Response for more info
# "data":[] # an array containing trade information and a timestamp
#
# }

print(msg)

subscribe = {
'eventName':'subscribe',
'authorization':'API_KEY_GOES_HERE',
#see https://api.tiingo.com/documentation/websockets/iex > Request for more info
'eventData': {
'thresholdLevel':5
}
}
# notice how the object isn't needed after using it
# any logic should be implemented in the callback function
TiingoWebsocketClient(subscribe,endpoint="iex",on_msg_cb=cb_fn)
while True:pass


Further Docs
--------
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def read(*filenames, **kwargs):

requirements = [
'requests',
'websocket-client'
]

setup_requirements = [
Expand Down
42 changes: 42 additions & 0 deletions tests/test_wsclient.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from unittest import TestCase
from tiingo.wsclient import TiingoWebsocketClient
from tiingo.exceptions import MissingRequiredArgumentError

class TestRestClientWithSession(TestCase):
def setUp(self):

def msg_cb(msg):
print(msg)

self.cb=msg_cb

self.config = {
'eventName':'subscribe',
'authorization':'API_KEY_GOES_HERE',
#see https://api.tiingo.com/documentation/websockets/iex > Request for more info
'eventData': {
'thresholdLevel':5
}
}

# test for missing or incorrectly supplied endpoints
def test_missing_or_wrong_endpoint(self):
with self.assertRaises(AttributeError) as ex:
TiingoWebsocketClient(config=self.config,on_msg_cb=self.cb)
self.assertTrue(type(ex.exception)==AttributeError)

with self.assertRaises(AttributeError) as ex:
TiingoWebsocketClient(config=self.config,endpoint='wq',on_msg_cb=self.cb)
self.assertTrue(type(ex.exception)==AttributeError)

# test for missing API keys in config dict
def test_missing_api_key(self):
with self.assertRaises(RuntimeError) as ex:
TiingoWebsocketClient(config={},endpoint='iex',on_msg_cb=self.cb)
self.assertTrue(type(ex.exception)==RuntimeError)

# test for missing callback argument
def test_missing_msg_cb(self):
with self.assertRaises(MissingRequiredArgumentError) as ex:
TiingoWebsocketClient(config=self.config,endpoint='iex')
self.assertTrue(type(ex.exception)==MissingRequiredArgumentError)
1 change: 1 addition & 0 deletions tiingo/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# -*- coding: utf-8 -*-
from tiingo.api import TiingoClient
from tiingo.wsclient import TiingoWebsocketClient

__author__ = """Cameron Yick"""
__email__ = '[email protected]'
110 changes: 110 additions & 0 deletions tiingo/wsclient.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
import os
import websocket
#to import the correct version of thread regardless of python version
try:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering, what is this try-except import for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to import the correct version of thread regardless of python version

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, that makes sense. Could we add a comment explaining this is the reason?

import thread
except ImportError:
import _thread as thread
import json
from tiingo.exceptions import MissingRequiredArgumentError

GLOB_config=None
GLOB_on_msg_cb=None

class genericWebsocketClient:
'''
the methods passed to websocketClient have to be unbounded if we want WebSocketApp to pass everything correctly
see websocket-client/#471
'''
def on_message(ws, message):
GLOB_on_msg_cb(message)
def on_error(ws, error):
print(error)
def on_close(ws):
pass
def on_open(ws):
def run(*args):
print(GLOB_config)
ws.send(json.dumps(GLOB_config))
thread.start_new_thread(run, ())
def __init__(self,config,on_msg_cb):
global GLOB_config
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for investigating this! I'm a little bit concerned by the use of global here, so I went and looked at the issue in the comment, and I'm not sure that I entirely follow why resolving this requires using globals. websocket-client/websocket-client#471

I was wondering if there were any existing Websocket clients that this implementation was inspired by that I could take a look at.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have tracked this down to this line of code in the library.
which checks if the method is bounded before passing self to it which is needed for the number of arguments to match up. and ofc you can't have unbounded method access class data. I would love not to use global ,but i have no idea how to implement something like that without actually changing the websocket-client code. if you have an idea or if i am missing something, Please. let me know

Copy link
Owner

@hydrosquall hydrosquall Sep 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unclear about why genericWebsocketClient class is necessary in the first place, since we never actually even read the value in the ws_client variable. This is why I was curious if this is based on an existing production library / program that you saw, so I can evaluate whether that approach makes sense for us too.

If we're not actually going to read the value of ws_client, I'm not sure that I see the value of creating it in the first place.

global GLOB_on_msg_cb
GLOB_config=config
GLOB_on_msg_cb=on_msg_cb
return

class TiingoWebsocketClient:
'''
from tiingo import TiingoWebsocketClient

def cb_fn(msg):

# Example response
# msg = {
# "service":"iex" # An identifier telling you this is IEX data.
# The value returned by this will correspond to the endpoint argument.
#
# # Will always return "A" meaning new price quotes. There are also H type Heartbeat msgs used to keep the connection alive
# "messageType":"A" # A value telling you what kind of data packet this is from our IEX feed.
#
# # see https://api.tiingo.com/documentation/websockets/iex > Response for more info
# "data":[] # an array containing trade information and a timestamp
#
# }

print(msg)

subscribe = {
'eventName':'subscribe',
'authorization':'API_KEY_GOES_HERE',
#see https://api.tiingo.com/documentation/websockets/iex > Request for more info
'eventData': {
'thresholdLevel':5
}
}
# notice how the object isn't needed after using it
# any logic should be implemented in the callback function
TiingoWebsocketClient(subscribe,endpoint="iex",on_msg_cb=cb_fn)
while True:pass
'''

def __init__(self,config={},endpoint=None,on_msg_cb=None):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about documenting the expected.permitted options for the config dictionary + the expected values for endpoint + the function signature for on_msg_cb ? The example code you provided in the docstring is a good start for a documentation page, but we would want to aim for something closer to a descriptive docstring for a class comment. See https://realpython.com/python-pep8/#documentation-strings for more background.


self._base_url = "wss://api.tiingo.com"
self.config=config
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're getting a warning from LGTM about mutating a basic object. I think we can solve it by doing

https://lgtm.com/rules/4840097/

  • Make the initial value None in the init constructor statement
self.config = {} if config is None else config

This should address the warning.


try:
api_key = self.config['authorization']
except KeyError:
api_key = os.environ.get('TIINGO_API_KEY')
self.config.update({"authorization":api_key})

self._api_key = api_key
if not(api_key):
raise RuntimeError("Tiingo API Key not provided. Please provide"
" via environment variable or config argument."
"Notice that this config dict takes the API Key as authorization ")

self.endpoint = endpoint
if not (self.endpoint=="iex" or self.endpoint=="fx" or self.endpoint=="crypto"):
raise AttributeError("Endpoint must be defined as either (iex,fx,crypto) ")

self.on_msg_cb = on_msg_cb
if not self.on_msg_cb:
raise MissingRequiredArgumentError("please define on_msg_cb It's a callback that gets called when new messages arrive "
"Example:"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea to put an example function in the error message for how to fix the bug, developers really appreciate that! :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glad you liked it

"def cb_fn(msg):"
" print(msg)")

ws_client = genericWebsocketClient(config=self.config,on_msg_cb=self.on_msg_cb)


websocket.enableTrace(True)

ws = websocket.WebSocketApp("{0}/{1}".format(self._base_url,self.endpoint),
on_message = genericWebsocketClient.on_message,
on_error = genericWebsocketClient.on_error,
on_close = genericWebsocketClient.on_close,
on_open = genericWebsocketClient.on_open)
ws.run_forever()