-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: build with config.gypi from node headers #2497
Conversation
8384907
to
d1b2c9c
Compare
Given that node-sass is deprecated, can we take it out of the equation? |
Yeah node-sass is deprecated but the problem is still there. For example this FreeBSD bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220983, the stock Node.js binary provided in FreeBSD basically can not work with any C++ native modules. For the FreeBSD bug, the eventual solution would require them to provide their own node headers distributions (same with what Electron and NW.js have been doing), and node-gyp needs this change to fully support that use case, otherwise there will be more downstream patches on Node.js and node-gyp. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
I think the pointer stands and we have flags to control. LGTM. @rvagg Second opinion ? |
// 1. string comments | ||
config = config.replace(/#.*/g, '') | ||
// 2. join multiline strings | ||
config = config.replace(/'$\s+'/mg, '') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we have any examples of this in config.gypi? I'm looking but can't see any, maybe this is safe to include but it's a bit of an uncomfortable regex that would be nice to see tested against an example (even if just manually)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A test was added for this in test/test-create-config-gypi.js
:
const str = "# Some comments\n{'variables': {'multiline': 'A'\n'B'}}"
const config = parseConfigGypi(str)
t.deepEqual(config, { variables: { multiline: 'AB' } })
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I suppose we can give it a go! it at least seems to preserve existing behaviour, only adding something new
This might fix #2534. |
d1b2c9c
to
6081647
Compare
6081647
to
70aa919
Compare
In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]> Co-authored-by: Darshan Sen <[email protected]>
In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]> Co-authored-by: Darshan Sen <[email protected]>
fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Darshan Sen <[email protected]>
fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Co-authored-by: trop[bot] <37223003+trop[bot]@users.noreply.github.com> Co-authored-by: Darshan Sen <[email protected]>
fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
fix: recommended `node-gyp` version in `node.h` error (#37829) fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
fix: recommended `node-gyp` version in `node.h` error (#37829) fix: recommended node-gyp version in node.h error In https://github.com/electron/electron/blob/main/docs/tutorial/using-native-node-modules.md#using-npm, we recommend setting the `npm_config_disturl` variable but doing that does not work on node-gyp v8.4.0 because after nodejs/node-gyp#2497 landed, the dist URL was read only from `gyp.opts['dist-url']`. The fix for reading the value from `npm_config_disturl` by parsing `gyp.opts.disturl` was landed in nodejs/node-gyp#2547 and that change was released in node-gyp v9.0.0, so this change updates the error macro to recommend node-gyp v9.0.0 as the minimum required version. Signed-off-by: Darshan Sen <[email protected]>
Checklist
npm install && npm test
passesDescription of change
Close #2490.
This PR makes the generated
config.gypi
file inherit from$nodedir/include/node/config.gypi
instead of runtime'sprocess.config
object when--nodedir
or--dist-url
is passed. There is also a--force-process-config
flag added to force switching back to the old behavior. For the reason behind this change, please check #2490.Affects of change
This change should have not affect to Node.js users, since they usually don't use either
--nodedir
or--dist-url
. And even when they explicitly do so, since theprocess.config
is generated from the$nodedir/include/node/config.gypi
, the build configurations will be exactly the same.For users building modules for Electron, this is a breaking change because old versions of Electron are shipping a malformed
config.gypi
in its headers distribution, and users have to pass--force-process-config
to node-gyp to correctly build modules.For NW.js users I believe nothing is changed as they are using a custom fork of node-gyp to build modules.
--nodedir
and--dist-url
Note that
$nodedir/include/node/config.gypi
is only used when--nodedir
or--dist-url
are passed.There are 3 kinds of node-gyp use cases:
For 1, using either
$nodedir/include/node/config.gypi
orprocess.config
is fine since they are the same thing.For 2, using
$nodedir/include/node/config.gypi
can cause problems, because some third party Node.js distributions are using different build configurations from the official headers.For example, some Node.js distributions provided in Linux and BSD distributions are using shared libuv and openssl library, and building with
$nodedir/include/node/config.gypi
would cause problems for them because the$nodedir/include/node/config.gypi
file in official headers distribution assumes bundled libuv and openssl.This is actually a malformed use case because there is no guarantee that official headers can build with custom
process.config
settings, and it has been causing problems like sass/node-sass#2775. Third party Node.js distributions should build modules with their own headers instead of the official headers, there is no way for Node.js and node-gyp to provide an official build configuration that can work for all kinds of third party Node.js variants. But for now I think we should just make things compatible.For 3, users have to build modules with
$nodedir/include/node/config.gypi
instead ofprocess.config
, because only the former includes the correct build configurations of the target to build for. Electron has been patching Node.js to ignore the wrong build configurations used in node-gyp, and NW.js is just forking node-gyp.So making node-gyp use
$nodedir/include/node/config.gypi
when--nodedir
or--dist-url
is passed is a natural choice, because passing the flags means building with custom headers, and it implies that the build configurations of the custom target should be used.And when
--nodedir
or--dist-url
is not passed, node-gyp should just keep the old behavior so it won't break the use case 2.Why this is needed
Without this change node-gyp always builds modules with the build configurations of the running Node instance, and custom Node distributions like Electron and NW.js have been forking either Node.js or node-gyp to be able to build modules.
By making node-gyp use the
config.gypi
in node headers, embedders will be able to get rid of their patches, and the ecosystem will have less variants./cc @rvagg @nodejs/embedders