Define "Node.js core"

[rvagg]

Action item from the board meeting today: we need to define what core is, when we operate on this area we have a certain amount of independence but we all need to be clear on where that is and need to come up with language to describe it. As we've already noted, this is hard, but we need to do this work.

[MylesBorins]

Here's a stab

• c++ layer that handles I/O, callbacks / async management, and the JS vm
• infrastructure to glue together c++ > js
• Standard set of modules that allow for basic functionality on top of c++ layer

[cjihrig]

How about just the contents of nodejs/node? You can build that code and run the core platform.

[Fishrock123]

@cjihrig but we also need to make sure node-gyp and NAN work, and also the build infrastructure.

[fivdi]

How about just the contents of nodejs/node?

@cjihrig That's a nice definition because it's easy to understand. Maybe the contents of nodejs/node except for the contents of nodejs/node/deps would be better?

• c++ layer that handles I/O, callbacks / async management, and the JS vm
• infrastructure to glue together c++ > js
• Standard set of modules that allow for basic functionality on top of c++ layer

@thealphanerd A definition similar to this may be a too abstract for some/many even if it's a good definition. In other words, I think I know what the definition means but I'm not 100% sure.

On the other hand, perhaps both a concrete and an abstract definition are needed. Defining Node.js as the contents of nodejs/node may not sound the best but it's concrete and easy to understand. It's probably not a definition that would be displayed above the fold at https://nodejs.org. This is where an abstract definition is more appropriate.

[Fishrock123]

I suppose core is nodejs/node plus other tooling that needs to be maintained in direct sync with the same team to make sure that the rest of the ecosystem work.

But that's kinda broad. node-serialport would also fall under that. Then again, that's why we've talked about bringing serialport into core, so perhaps not a bad measure.

[nebrius]

There's also libuv and http parser which may or may not be considered part of core, I would favor including those types of project when considering a general definition, even if they are independent projects.

But that raises the question of how do we want to consider projects outside of nodejs/node, and at what level of entanglement do we say "ok, this is part of core?" OpenSSL is also a dependency, like libuv, but do we also consider it part of core? Probably not.

[Fishrock123]

is v8 part of core? Dx

[MylesBorins]

I think a good thing to add is that core is made up of the parts of the Node.js ecosystem that the collaborators are responsible for maintaining and keeping operational.

This would include

• nodejs/node
• nan
• node-gyp
• Build Infra
• Official Docker images

But may not include

• libuv
• v8
• OpenSSL

If the team decides that nodejs/collaborators should be responsible for a part of the ecosystem, that could be deemed "core", such as node-serialport

So now I'm left with the question... is core the codebase or the ecosystem

[Fishrock123]

As a note, I don't think libuv is part of core per-se, but we have to maintain it anyways.

[cjihrig]

I would argue that these things are not part of core. I can run a hello world application with NAN. They are just REALLY important tools, but not core. Also, libuv and the other deps are not part of core, but the versions living in the core repo are because you can't build without them.

[nebrius]

So now I'm left with the question... is core the codebase or the ecosystem

Aye, there's the rub. Given that this definition ties into the discussions of the TSC's scope, I would favor a definition that leans more towards the ecosystem side of things, although there does need to be a limit. A very broad interpretation of "the ecosystem" could even include everything in npm, which of course is overly broad.

I would recommend something along the lines of "Node and the things necessary to build the Node distribution package", but with some concept of "Spheres of Influence" mixed in to temper cases such as OpenSSL.

[fivdi]

I just took a look at the dictionary

core - the central, innermost, or most essential part of anything.
ecosystem - a system, or a group of interconnected elements, formed by the interaction of a community of organisms with their environment.

Is the central, innermost, or most essential part of Node.js the ecosystem?

[mhdawson]

Along the lines of the last comment, given that the discussion is with respect to scope I think we are better to look at whether we think the TSC should be making independent decisions as opposed to where the code came from. For example the scope of Node core should include the components for providing the ssl functionality, the TSC has just made the decision at this point that core will use OpenSSL for that component.

I think this is line with what @cjihrig said above in the libuv case. Core does not necessarily include the libuv project, but it does cover the components we chose to provide the async even functionality.

Maybe:

Node core is the set of components that are required to provide the APIs provided by the Node.js runtime.

Being part of core does not mean the component is necessarily fully under the TSC control (ex v8) but that the TSC should have decision making over how that component (or which components) are used within the runtime. For example it is the TSC decision that the current javascript runtime is v8.

We might be challenged that this is too broad as we can just expand the API to increase scope.

[joshgav]

I've been thinking about this deeply for a little while. Following is my understanding of what comprises the core Node platform from an architectural perspective, similar to @thealphanerd above. At the end of this note I describe Node's bootstrap process, which provides proof for the architectural description.

Accepting an architectural description will enable us to determine if and where a component should be included in the core Node platform, current governance ignored. Components to be considered include native libs such as libuv, cares, and v8; and JS modules such as fs, http, and ws; I intentionally include some which are under lesser degrees of this project's influence, all of which could nevertheless be considered part of the core Node architecture.

The Node.js Platform includes:

1. A host for a JSVM (i.e. v8::Isolate) and an event loop (i.e. uv_loop_t). This host hooks the JSVM to enable proxying JS calls to native libraries and vice versa. It is implemented as an intertwined native (C++) node::Environment instance and JS global process object.

   Proxying between JS and native is the main feature Node adds which wouldn't be possible through command-line invocation of a JSVM, a la V8's D8 or SpiderMonkey's JavaScript Shell.

   The PURPOSE of hosting and adapting a JSVM (i.e. V8) is the second part of the core Node platform:

2. A standard collection of "native" modules essential for a computing platform outside of a browser and not previously available in the browser-centric JSVMs, such as fs, net, http, etc.

---

It seems the true goal at least originally was the 2nd bullet and the 1st was simply a means. As I understand it, @ry was attracted to the JSVM/V8 because the lack of existing standard JS libraries coupled with JS's natural event-driven orientation provided an ideal opportunity to freshly, in a greenfield, create those standard libraries with an event-oriented API.

A perhaps less-expected but quite important additional result was revealing industry interest in using JSVMs outside of browsers.

---

The Node bootstrap process reveals the Node platform architecture as described above. Following are the main steps of that process. All line numbers are based on this or nearby commits.

Step	Description	Source Location
1	startup: main --> node::Start --> node::StartNodeInstance	./src/node_main.cc::51, ./src/node.cc::4345, ./src/node.cc::4383
2	create JSVM (v8::isolate)	./src/node.cc::4372, ./src/node.cc::4261
3	create event loop (uv_loop_t)	./src/node.cc::4377
4	create native environment (node::Environment) containing JSVM and event loop	./src/node.cc::4276, ./src/node.cc::4181
5	create JS process object bound to native environment	./src/node.cc::4237-4244
6	initialize JS context and process object through JS script	./src/node.cc::4289, ./src/node.cc::3330, ./lib/internal/bootstrap_node.js
7	invoke user script (or repl, --eval, etc.)	./lib/internal/bootstrap_node.js::66

[mikeal]

So, one of the things that has been hard for people outside of our bubble to understand is where the line is for "Node.js Core."

A common suggestion is that what "ships with Node.js" is Core. This is insufficient in two ways:

• We ship bundled dependencies, some of which we control and some that we don't, each with scope beyond just core: npm, libuv, v8.
• The "core platform" requires a few higher level tools that don't ship with core: node-gyp, nan.

We can call each of these things out but what has made messaging this difficult is finding an objective definition of the line in both what ships and what doesn't ship with core that is considered part of the core platform.

[robbie-mac]

If you are looking for a suggestion, mine would be this:
Anything that is required to achieve raw IO on the bare metal. Everything else is superfluous.

What I mean by this, is anything to achieve raw IO streams. That should be core, everything else, is not.

My list of what would be core are this:
Anything that is needed to achieve raw IO: Streams and Buffer
Anything that directly interacts with Node process and the OS: Process, Child Process, OS, V8, Gyp, LibUV...
Everything else is part of node, but not core/central to it.

Just my 2 cents

[williamkapke]

An attempt to create a generic statement that accurately defines the scope of Node.js Core hasn't surfaced yet. I think it is an indication that the approach isn't working. I see positive reactions to collaborators listing things out... so, how about creating a Scope document that defines it instead?

It seems there are many things that the TSC & CTC find to be clearly within scope of Node.js Core and some that are borderline (grey areas). Progress could be made by starting to add in the things that are undisputed and then have breakout conversation/votes for the grey areas. Additions/Removals could be made from time to time as needed.

While a generic statement allows agility & flexibility (which I think is the desire), I fear the ambiguity will continue to plague progress in this task and in future decision making of each scope.

[mhdawson]

See #113 for an attempt to make progress on this.

[jasnell]

Closing given that there's been no further discussion and several PRs have landed that refine this.