Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add baseline enforcement within the runtime #140

Merged
merged 2 commits into from
Nov 3, 2022

Conversation

vlovich
Copy link
Contributor

@vlovich vlovich commented Oct 28, 2022

Middleware might restrict the size of the URL that would get generate. Set a sane unrealistic upper bound that's guaranteed to get through to the KV worker. This should make error messages reliably clear when a name provided is too long.

@vlovich vlovich force-pushed the vlovich/limit-kv-key-length branch 2 times, most recently from bdf232b to bb0e860 Compare October 28, 2022 18:14
@koeninger
Copy link
Contributor

Isn't this a breaking change unless it's literally exactly the same error text that's currently thrown by the KV worker for keys in the 512b - 32k range?

@vlovich
Copy link
Contributor Author

vlovich commented Oct 28, 2022

I had a mild concern that the message would be different (hence why I initially chose 8 kib - the probability of you seeing it is minimized unless you're really going crazy). If this is a concern I'd rather go back to 8 kib or even 32 kib limit here rather than trying to massage the error message to be identical as that seems harder to accomplish / harder to test for. What's not clear to me is how it's a breaking change - an error is an error and we have never bothered with error message stability as a back compat guarantee.

Thinking about this some more, the bigger breaking change concern I have here is that this:

NAMESPACE.get(<very long string>)

will now throw whereas before you'd need to await because it was coming from the KV worker:

NAMESPACE.get(<very long string>).catch(...)

I can add a separate js.evalNow to wrap the entrypoint functions because it looks like the KV runtime hadn't been doing that whereas @kentonv had indicated all entrypoint methods returning a promise should be doing that, but I want to get confirmation that we want to go ahead and do that. Not sure the consequence of that as it will subtly change some of the exception generation but historically IIRC we've just gone ahead and done that without a compat flag.

src/workerd/api/kv.c++ Outdated Show resolved Hide resolved
@bretthoerner
Copy link
Contributor

What's not clear to me is how it's a breaking change - an error is an error and we have never bothered with error message stability as a back compat guarantee.

Dunno what to say, we've just been careful about it in KV. For better or worse the message includes an HTTP status code and there is definitely code in the wild that string matches on them. I do agree that it'd be pretty weird for someone to catch long key lengths, when they could easily check that in JS before calling KV methods, but 🤷

But yeah, throwing synchronously and not after an await feels like a bigger change.

src/workerd/api/kv.c++ Outdated Show resolved Hide resolved
@vlovich vlovich force-pushed the vlovich/limit-kv-key-length branch 3 times, most recently from 7786a48 to 17fa9da Compare November 2, 2022 17:54
@vlovich
Copy link
Contributor Author

vlovich commented Nov 2, 2022

I think I've addressed the comments.

  1. Extracted out constant to a standalone variable.
  2. Mimic the error response to be similar to what it would look like if FL rejected it.
  3. Wrap the code in js::evalNow to fix the preexisting issue that exceptions were thrown at the call site instead of within the promise.
  4. Internal PR adds a test for the new error message.

@koeninger
Copy link
Contributor

similar to what it would look like if FL rejected it.

It's not FL rejecting it that's the common case for 512b < size < 32k, so it's sgw's error message that we care about being the same as, right?

Copy link
Contributor

@bretthoerner bretthoerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may as well match up the error since it's easy, but LGTM.

src/workerd/api/kv.c++ Outdated Show resolved Hide resolved
@vlovich vlovich force-pushed the vlovich/limit-kv-key-length branch 2 times, most recently from 87557b9 to 537fb2f Compare November 3, 2022 15:40
Middleware might restrict the size of the URL that would get generate.
Set a sane unrealistic upper bound that's guaranteed to get through to
the KV worker. This should make error messages reliably clear when a
name provided is too long.
The runtime has a consistent style of throwing validation exceptions for
asynchronous function when the promise is resolved rather than at the
time of construction. While technically an observable change,
we've never created compat flags for it in the past.
@vlovich vlovich force-pushed the vlovich/limit-kv-key-length branch from 537fb2f to ee79778 Compare November 3, 2022 15:41
@vlovich vlovich merged commit f7ed1b7 into main Nov 3, 2022
@vlovich vlovich deleted the vlovich/limit-kv-key-length branch November 3, 2022 15:42
@kentonv
Copy link
Member

kentonv commented Dec 7, 2022

FYI, there is actually a compat flag which causes JSG to convert synchronous exceptions into async exceptions across all APIs that return promises, which became default 2022-10-31. So the introduction of evalNow() in this PR is a noop for anyone who has that flag set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants