Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support decoding user-marshalled objects #30

Merged
merged 4 commits into from
Jul 28, 2022

Conversation

frerich
Copy link
Contributor

@frerich frerich commented May 20, 2022

This PR implements two enhancements which permit decoding user-marshalled objects (indicated by the U type code):

  1. Whenever encountering a user-defined object (e.g. a Ruby date) in a stream, the code now emits a 2-tuple giving the object's Ruby class name (a symbol, e.g. Date) as well as any attributes associated with that class (as written out by the marshal_dump method of the Ruby class). For example, here's the result of parsing the data marshalled by running Marshal.dump(Date.today) in Ruby:
iex(1)> value = <<4, 8, 85, 58, 9, 68, 97, 116, 101, 91, 11, 105, 0, 105, 3, 72, 136, 37, 105, 0, 105, 0, 105, 0, 102, 12, 50, 50, 57, 57, 49, 54, 49>>
<<4, 8, 85, 58, 9, 68, 97, 116, 101, 91, 11, 105, 0, 105, 3, 72, 136, 37, 105,
0, 105, 0, 105, 0, 102, 12, 50, 50, 57, 57, 49, 54, 49>>
iex(2)> ExMarshal.decode(value)
{:Date, [0, 2459720, 0, 0, 0, 2299161.0]}
  1. ExMarshal.decode (and ExMarshal.Decoder.decode) now support taking options. The only supported option is called user_object_parsers which is expected to take a map associating Ruby class names (i.e. Elixir atoms) with unary functions processing the attributes written by Ruby. This is useful to customise the handling of user objects such that instead of getting a 2-tuple as shown above, custom Ruby types like Date get automatically converted to appropriate Elixir types. Here's an example showing how it can be used to convert Ruby dates to Elixir dates:
value = ...
iex(3)> ExMarshal.decode(value, user_object_parsers: %{Date: fn [_, julian_day, _, _, _, _] -> Date.from_gregorian_days(julian_day - 1721425) end})
~D[2021-05-20]

Two plausible follow-up improvements come to mind:

  • I'm not a ExMemcached user myself, but I suppose it would be plausible to also permit specifying custom parsers via configuration files such that the transcoder of Memcached would transparently map types using user object parsers.
  • A new module ExMarshal.CustomParsers or such could be devised which implements custom parser functions for common Ruby types such as Date.

frerich added 4 commits May 20, 2022 09:07
I plan to extend the state with a new field. Let's make sure that this
field gets passed around as expected by making update_references not
build a brand new state from scratch. Instead, we can just update a
single field.
This isn't used for anything yet, but I plan to pass additional flags to
decode() which can be used to influence how user objects are decoded.
This option can be used to specify a map associating atoms with unary
functions. These functions will be considered when trying to decode user
objects. I.e. instead of user objects resulting in 2-tuples {class_name,
attributes}, a custom function is invoked which can then do something
interesting with the attributes.

One example for this would be to parse a Ruby date into an Elixir date.
@gaynetdinov
Copy link
Owner

Hey, sorry for the late reply.
I will merge and release the new version this Friday.
Thanks for your contribution!

@gaynetdinov gaynetdinov merged commit e8de523 into gaynetdinov:master Jul 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants