Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[document_database] add custom json encoder for TinyDB #73

Closed
signebedi opened this issue Mar 27, 2024 · 1 comment
Closed

[document_database] add custom json encoder for TinyDB #73

signebedi opened this issue Mar 27, 2024 · 1 comment

Comments

@signebedi
Copy link
Owner

signebedi commented Mar 27, 2024

Right now, we pass data to the document database object as json, which it then converts back to a dict before converting back to json. The reason for this is because the pydantic encoder DOES support serialized date and datetime objects, inter alia, but the default JSON library does not. But this degree of data conversion seems a little hackish. Instead, we should extend tinydb to use either pydantic's encoding method OR a custom encoder. The developer of tinydb does have a comment on this in the issues, where he recommends subclassing the JSONStorage and writing our own custom logic...

I defenitely think it's useful but I'm not sure how to implement this best. I wouldn't make this a default, though, because that would break usages where a conversion is not expected. Instead one could make use of TinyDB's extensibility. At the moment I can think of these options:

  • Create a custom storage (copying the source of JSONStorage) that uses a custom JSON encoder/decoder
  • Implement the encoder/decoder as a middleware. It would scan the data for datetime objects when writing and for date-like strings when reading and convert between these two

Originally posted by @msiemens in msiemens/tinydb#48 (comment)

@signebedi
Copy link
Owner Author

I am tracing through TinyDB, and it looks like the TinyDB class will pass kwargs to the JSONStorage class at instantiation. So, an easier way to do this might be to create a custom encoder and pass the cls=CustomEncoder kwarg, which will make it all the way to the json.dumps calls in the JSONStorage object ....

I'll try doing this first to avoid creating too much custom tinydb code that we need to manage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant