Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add serialization support for Client and Dataset #293

Merged
merged 32 commits into from
Jan 28, 2023

Conversation

billschereriii
Copy link
Contributor

@billschereriii billschereriii commented Jan 24, 2023

C, C++, Fortran get to_string() methods
C++ also gets ostream& operator<<() methods
Python gets str() methods

Sample output (Client):

>>> from smartredis import Client
>>> c = Client("127.0.0.1:6381", cluster=True, logger_name="testcase"); print(c)
Client (testcase):
Clustered Redis connection:
  Redis shards at:
    127.0.0.1:6383
    127.0.0.1:6381
    127.0.0.1:6382
  Protocol: TCP
  Command parameters:
    Retry attempts: 101
    Retry interval (ms): 1000
    Attempt timeout (ms): 100
  Connection parameters:
    Retry attempts: 101
    Retry interval (ms): 1000
    Attempt timeout (ms): 100
  Threadpool: 4 threads

Sample output (DataSet):

>>> from smartredis import Dataset
>>> import numpy as np
>>> d = Dataset("This is a test")
>>> data = np.uint8([2,4,8])
>>> d.add_tensor("u8_tensor", data)
>>> data = np.double([2.0,4.1,8.3, 5.6])
>>> d.add_tensor("double_tensor", data)
>>> d.add_meta_scalar("float2_scalar", np.float(3.1415926535))
>>> d.add_meta_scalar("float_scalar", np.double(3.1415926535))
>>> d.add_meta_string("metastring", "metavalue")
>>> print(d)
DataSet (This is a test):
Tensors:
  double_tensor:
    type: double
    dimensions: [4]
    elements: 4
  u8_tensor:
    type: 8 bit unsigned integer
    dimensions: [3]
    elements: 3
Metadata:
  metastring:
    type: string
  float2_scalar:
    type: double
  float_scalar:
    type: double

@billschereriii billschereriii added area: test Issues related to the test suite area: python Issues related to the Python Client area: C++ Issues related to the C++ client area: C Issues related to the C client area: fortran issues related to the Fortran Client labels Jan 24, 2023
@billschereriii billschereriii self-assigned this Jan 24, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jan 26, 2023

Codecov Report

Merging #293 (b5c2a93) into develop (a90f051) will decrease coverage by 0.65%.
The diff coverage is 61.87%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #293      +/-   ##
===========================================
- Coverage    77.13%   76.48%   -0.65%     
===========================================
  Files           66       66              
  Lines         4151     4266     +115     
===========================================
+ Hits          3202     3263      +61     
- Misses         949     1003      +54     
Impacted Files Coverage Δ
include/metadata.h 100.00% <ø> (ø)
include/metadatafield.h 100.00% <ø> (ø)
include/redisserver.h 0.00% <ø> (ø)
include/scalarfield.h 100.00% <ø> (ø)
include/srobject.h 11.11% <ø> (ø)
include/stringfield.h 100.00% <ø> (ø)
include/tensorpack.h 100.00% <ø> (ø)
src/cpp/utility.cpp 55.05% <10.52%> (-33.18%) ⬇️
src/cpp/dataset.cpp 73.71% <61.53%> (-5.15%) ⬇️
src/cpp/redis.cpp 47.90% <80.00%> (+0.48%) ⬆️
... and 14 more

@billschereriii billschereriii marked this pull request as ready for review January 26, 2023 00:26
Copy link
Member

@MattToast MattToast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked through the *.py/pybind/Py*.cpp files in detail and have a couple of small notes, but otherwise everything looks good. Skimmed through the reset and didn't see any immediate red flags.

Overall looks great!! This should be very helpful for users attempting to debug their simulations!!

Copy link
Collaborator

@ashao ashao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great and again, a great quality-of-life improvement. Feel free to ignore the comment about passing by value. However, I would request that you change the write unit to something something other than STDERR. Those are two-line fixes and don't need to re-review. Thanks @billschereriii!

@MattToast MattToast self-requested a review January 27, 2023 00:22
Copy link
Member

@MattToast MattToast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!

Copy link
Collaborator

@mellis13 mellis13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor question/suggestion. Otherwise, great addition to the library and LGTM!

@billschereriii billschereriii merged commit 86b6d7a into CrayLabs:develop Jan 28, 2023
@billschereriii billschereriii deleted the stringy branch January 28, 2023 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: C++ Issues related to the C++ client area: C Issues related to the C client area: fortran issues related to the Fortran Client area: python Issues related to the Python Client area: test Issues related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants