Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new inspect submodule #253

Merged
merged 6 commits into from
Jan 5, 2023
Merged

Add new inspect submodule #253

merged 6 commits into from
Jan 5, 2023

Conversation

jcrist
Copy link
Owner

@jcrist jcrist commented Jan 2, 2023

This adds a new inspect submodule. The main entrypoint in this submodule is msgspec.inspect.type_info which converts a python type annotation into a msgspec.inspect.Type tree. This can be useful for tools that want to programmatically work with msgspec type annotations. A few possible use cases:

  • Generating openapi documentation
  • Generating fake data from type annotations

Fixes #227.

@jcrist
Copy link
Owner Author

jcrist commented Jan 2, 2023

So far this is just a skeleton outlining the types and functions, the actual implementation still needs to be filled in.

@Goldziher can you take a look through the types in msgspec.inspect to see if this information satisfies your needs?

@jcrist jcrist force-pushed the inspect branch 2 times, most recently from 98c5218 to 6bd7b1c Compare January 3, 2023 17:17
@jcrist
Copy link
Owner Author

jcrist commented Jan 3, 2023

Ok, the code here is now mostly done (caveat - still needs tests). I've ported our builtin json-schema generation code over to consume the output of msgspec.inspect.multi_type_info. There's a bit of mucking, but overall it feels pretty clean to use. You can see that code in the _json_schema.py file - the bulk of it is a big if-else statement in _to_schema.

Remaining todos:

  • Docs
  • Tests

jcrist added 4 commits January 3, 2023 23:38
This adds a new `inspect` submodule. The main entrypoint in this
submodule is `msgspec.inspect.type_info` which converts a python type
annotation into a `msgspec.inspect.Type` tree. This can be useful for
tools that want to programmatically work with msgspec type annotations.

A few possible use cases:

- Generating `openapi` documentation
- Generating fake data from type annotations
Also moves all json-schema handling over to consume the output of
`msgspec.inspect.multi_type_info`.

Still needs tests and docs.
@jcrist jcrist changed the title WIP - add new inspect submodule Add new inspect submodule Jan 4, 2023
@jcrist
Copy link
Owner Author

jcrist commented Jan 4, 2023

Ok, I believe this to be done. I'm pretty happy with the code quality, and think this provides a nice enough interface to build further tooling off of.

I've marked this as "experimental" in the docs for now. Here this just means that I don't feel bad about making breaking changes to it between versions if issues turn up during use.

I plan to merge this tomorrow evening if no comment. I would love for some feedback here from @Goldziher or @provinzkraut if y'all have time, but no pressure.

@Goldziher
Copy link

Hi, sure - I'm flying today, so i might need a couple of days to go through this properly

Copy link

@Goldziher Goldziher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Left some questions. I didn't see handling of TypedDict. Also, hoe can I access the raw typing?

names.append(field.encode_name)
fields.append(field_schema)
if isinstance(t, mi.NamedTupleType):
schema["type"] = "array"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of repetition. I'd consider using enums or constants

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you expand on what you mean by that? How would enums or constants make this code more maintainable?

@jcrist
Copy link
Owner Author

jcrist commented Jan 4, 2023

I didn't see handling of TypedDict

Handling of TypedDict in which part?

  • TypedDictType definition is here
  • Conversion of a TypedDict to TypedDictType is here
  • Conversion of TypedDictType to a json-schema is here

Also, hoe can I access the raw typing?

You mean the original python type annotations? Those aren't exposed. Based on this comment:

The above is preceisely what I was asking not to do. I know how to access type hinting, or how to inspect data on a class. This though is highly inefficent, and I was asking for an easy way to get the data that is already there. In pydantic I can simply access the fields attributes, which stores a mapping of FieldInfo instances and includes all the information about the parsed typing data etc. Also note whats called sub_fields in pydantic, which basically makes fields into a sort of syntax tree for typing information.

it sounded like you didn't want access to the raw python type annotations, but instead wanted "a sort of syntax tree for typing information", which is what the functions and types in msgspec.inspect provide. To get the original type annotations you can use typing.get_type_hints on the class, is this insufficient?

@jcrist
Copy link
Owner Author

jcrist commented Jan 5, 2023

Merging this for now. Any additional information needed can be exposed in follow-up PRs.

@jcrist jcrist merged commit f9732e0 into main Jan 5, 2023
@jcrist jcrist deleted the inspect branch January 5, 2023 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Access fields data
2 participants