refactor(runtime,serverless,cli): parallel requests to the same isolate

[QuiiBz]

About

Following #616 where we allowed parallel requests between isolates, this PR allows parallel requests to the same isolate.

Events

Each isolate is now in its own thread, instead of having a pool of workers. The event loop is now constantly running for each isolate. They can receive two kinds of events: requests and termination.

Request events are added to a map of request id -> request context, and they are all polled in the event loop, and removed when the response has been sent. This is the heart of how parallel requests work. Termination events simply call v8::Isolate::TerminateExeuction.

Limits

Another thread is spawned inside the isolate to apply CPU time limits. At each event loop tick, a "heartbeat" is sent. If no heartbeat has been sent during the function's maximum CPU time, the isolate is terminated.

As before, a near heap limit callback (v8::Isolate::AddNearHeapLimitCallback) is set and terminates the isolate when called.

Misc

When an isolate is terminated, the correct termination result is sent for each ongoing request.

CPU time and memory usage are calculated after each response has been sent.

Performance

For a simple hello world:

export async function handler(request) {
  return new Response('Hello World!');
}

Before: ~77k req/s

After: ~82k req/s

[changeset-bot]

🦋 Changeset detected

Latest commit: 83ebcaf

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@lagon/cli	Patch
@lagon/runtime	Patch
@lagon/serverless	Patch
@lagon/js-runtime	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

4 Ignored Deployments

Name	Status	Preview	Comments	Updated
dashboard	⬜️ Ignored (Inspect)	Visit Preview		Mar 20, 2023 at 4:38PM (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		Mar 20, 2023 at 4:38PM (UTC)
storybook	⬜️ Ignored (Inspect)	Visit Preview		Mar 20, 2023 at 4:38PM (UTC)
www	⬜️ Ignored (Inspect)	Visit Preview		Mar 20, 2023 at 4:38PM (UTC)