"Critical dependency: the request of a dependency is an expression" with webpack, Babel & TypeScript

[arichter83]

Describe the bug
I am using a TypeScript setup with webpack and babel and get the following warnings when including elastic-apm-node.

WARNING in node_modules/elastic-apm-node/lib/agent.js 152:16-59
Critical dependency: the request of a dependency is an expression
 @ node_modules/elastic-apm-node/index.js

WARNING in node_modules/elastic-apm-node/lib/config.js 25:19-36
Critical dependency: the request of a dependency is an expression
 @ node_modules/elastic-apm-node/lib/agent.js

WARNING in node_modules/elastic-apm-node/lib/instrumentation/index.js 169:16-30
Critical dependency: the request of a dependency is an expression
 @ node_modules/elastic-apm-node/lib/agent.js

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/arichter83/meteor-webpack/tree/with-elastic-apm-node
2. Run npm install
3. Run npm start

How are you starting the agent? (please tick one of the boxes)

• Requiring elastic-apm-node/start from within the source code

[watson]

Are you trying to use the agent in a browser? If so, that's not supported. You might want to take a look at the JavaScript Real User Monitoring agent instead. It's been a while since I played around with Meteor, but if I recall correctly it allows you to write on app that runs both on the server and in the browser.

[vigneshshanmugam]

@arichter83 During bundling, every bundler(including webpack) generates module graph needed to basically decide what goes in to the bundle. When there are dynamic require calls used in the code, Webpack tries to produce warnings since those modules be accidentally included in the code. You can read more about the issue here - webpack/webpack#196

@watson We can simply use some tricks to fool webpack if users do use elastic-apm-node in universal(server & browser) environments. We use dynamic requires here and here.

[watson]

@vigneshshanmugam The Node.js agent use dynamic require-statements in all the 3 locations pointed out by the warnings. The first two is not strictly needed and can just be ignored (you can see that they are also enclosed in try-catch statements), but the last is required to run the agent (there's even a 4th not mentioned in the warnings, I guess because it uses a template sting).

But if we can be sure those warnings don't mean anything and we somehow silence them, I'd be all ears. I just imagine that there are many cases where the warnings are important if the user tries to compile the code to run in the browser, which obviously we can never support.

[vigneshshanmugam]

Thanks for the insights. You are right, template strings inside the require are usually ignored because they are computed as well.

I just imagine that there are many cases where the warnings are important if the user tries to compile the code to run in the browser, which obviously we can never support.

True, webpack warns because users might accidentally include the files on the browser which the code is never meant to run (like in our case). We can suppress warnings easily by either using template strings or using ideas like this https://github.com/getsentry/sentry-javascript/blob/bd35d7364191ebed994fb132ff31031117c1823f/packages/utils/src/misc.ts#L9-L11

[watson]

Not sure what's best though: Showing the warning to the user so they know not to run it in the browser, or trick Webpack by obfuscating the code so it doesn't show the warning... 🤔

[vigneshshanmugam]

I am also not sure which would be the best approach and users might also use the library to run it on the server side with webpack. we can probably wait and decide later after hearing more feedback from our users.

[techmunk]

Hi. I know this is old, but have some info that might be relevant.

The warning means webpack could not accurately determine exactly which files/modules to bundle, so it either a) bundled none of them, or b) bundled the ones it got "hints" about. (such as require(./modules/${patch}), where webpack will just include all files insides the ./modules` directory.

I've written a Webpack Plugin that allows elastic APM (more specifically require-in-the-middle) to work when used inside a webpack bundle. You can see the plugin at nodejs/require-in-the-middle#35 (comment) for reference. I've only tested this with Webpack 4. I might look at Webpack 5 soon.
We're bundling our entire server side application with webpack for quick docker deployments. Webpack build with full sourcemaps ~ 3MB, vs over 200MB for the entire of node_modules.

For the warnings, in Webpack 4 you can do the following in the webpack config to "silence" them.

this.config.stats = {
      // Filter out some warnings we don't want to be notified about.
      // The warnings filter removes messages pertaining to dynamic imports which cannot be resolved at build time.
      warningsFilter: (warning: string): boolean => {
        return warning.includes('Critical dependency: the request of a dependency is an expression') &&
          (
            warning.includes('log4js/lib/appenders/index.js') ||
            warning.includes('express/lib/view.js') ||
            warning.includes('keyv/src/index.js') ||
            warning.includes('elastic-apm-node/lib/agent.js') ||
            warning.includes('elastic-apm-node/lib/config.js') ||
            warning.includes('elastic-apm-node/lib/instrumentation/index.js')
          )
      }
    }

[havran]

Today I hit same issue with some additional problem. All related with latest Angular (10.x) and @nguniversal/express-engine. I try install clean Angular installation with following steps:

ng new angular-with-universal-express-and-elastic-apm
ng add @nguniversal/express-engine
npm install elastic-apm-node --save

Then i add to my server.ts:

import * as apm from 'elastic-apm-node/start';

apm.start({
  serviceName: process.env.ELASTIC_APM_SERVICE_NAME || `ferraweb-websites-dev`,
  serverUrl: process.env.ELASTIC_APM_SERVER_URL || 'localhost:8200',
  secretToken: process.env.ELASTIC_APM_SECRET_TOKEN,
});

When i try start npm run dev:ssr or npm run build:ssr, i get this errors:

$ npm run build:ssr

> [email protected] build:ssr /home/havran/tmp/angular-with-universal-express-and-elastic-apm
> ng build --prod && ng run angular-with-universal-express-and-elastic-apm:server:production


chunk {} runtime.acf0dec4155e77772545.js (runtime) 1.45 kB [entry] [rendered]
chunk {1} main.f213c4ef2e1009cc0c3e.js (main) 216 kB [initial] [rendered]
chunk {2} polyfills.35a5ca1855eb057f016a.js (polyfills) 36 kB [initial] [rendered]
chunk {3} styles.09e2c710755c8867a460.css (styles) 0 bytes [initial] [rendered]
Date: 2020-10-26T14:09:26.143Z - Hash: 6ccef84b74615d95690e - Time: 15782ms
Hash: f5cbfb84aaf23bf152c2
Time: 13039ms
Built at: 10/26/2020 3:09:42 PM
  Asset      Size  Chunks  Chunk Names
main.js  3.25 MiB       0  main
Entrypoint main = main.js
chunk    {0} main.js (main) 6.04 MiB [entry] [rendered]

WARNING in ./node_modules/elastic-apm-node/lib/agent.js 157:16-59
Critical dependency: the request of a dependency is an expression

WARNING in ./node_modules/elastic-apm-node/lib/config.js 546:13-30
Critical dependency: the request of a dependency is an expression

WARNING in ./node_modules/elastic-apm-node/lib/instrumentation/index.js 193:16-30
Critical dependency: the request of a dependency is an expression

ERROR in ./node_modules/elastic-apm-node/lib/agent.js
Module not found: Error: Can't resolve '../package' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/elastic-apm-node/lib'

ERROR in ./node_modules/elastic-apm-node/lib/config.js
Module not found: Error: Can't resolve '../package' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/elastic-apm-node/lib'

ERROR in ./node_modules/elastic-apm-http-client/index.js
Module not found: Error: Can't resolve './package' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/elastic-apm-http-client'

ERROR in ./node_modules/spdx-expression-parse/scan.js
Module not found: Error: Can't resolve 'spdx-exceptions' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/spdx-expression-parse'

ERROR in ./node_modules/spdx-correct/index.js
Module not found: Error: Can't resolve 'spdx-license-ids' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/spdx-correct'

ERROR in ./node_modules/spdx-expression-parse/scan.js
Module not found: Error: Can't resolve 'spdx-license-ids' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/spdx-expression-parse'

ERROR in ./node_modules/spdx-expression-parse/scan.js
Module not found: Error: Can't resolve 'spdx-license-ids/deprecated' in '/home/havran/tmp/angular-with-universal-express-and-elastic-apm/node_modules/spdx-expression-parse'
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build:ssr: `ng build --prod && ng run angular-with-universal-express-and-elastic-apm:server:production`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build:ssr script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/havran/.npm/_logs/2020-10-26T14_09_45_088Z-debug.log

[havran]

By use @angular-builders/custom-webpack and add custom webpack configuration, second part problems with errors solved:

const webpack = require('webpack');

module.exports = {
  resolve: {
    extensions: ['.json'],
  }
};

Now dev:ssr and build:ssr compile successfully.

[RedactedProfile]

Hitting the same problem as of today,

This happens during the build:ssr process

Unfortunately I do not know what @havran's workaround means

[astorm]

@RedactedProfile I'm not sure which part of @havran's work-around didn't make sense for you, so my apologies if I repeat things you already know.

The angular project uses webpack to compile its source code.

Angular has webpack configured to compile every module even if (like the Elastic Node.js Agent) they don't need to end up as part of the compiled frontend code.

Additionally, angular has webpack configured in such a way that this line confuses it. Webpack, as configured, doesn't know that const pkg = require('./package') is a request to load package.json.

(It's probably worth noting that when I say "angular has webpack configured" that some of these may be webpack's default configuration setting)

The @angular-builders/custom-webpack project looks like it's a project that will allow you to change angular's webpack configuration.

@havran's work-around shows the configuration they used to tell webpack that .json is a valid file extension for require.

So if I was in your shoes, I'd start with figuring out how to use @angular-builders/custom-webpack to add the configuration provided by @havran

[gtorre]

Big thanks to @havran your work around fixed the problem I was having!

[rickvandermey]

By use @angular-builders/custom-webpack and add custom webpack configuration, second part problems with errors solved:

const webpack = require('webpack');

module.exports = {
  resolve: {
    extensions: ['.json'],
  }
};

Now dev:ssr and build:ssr compile successfully.

is your application up to date?

I tried, and still got issues...
am i missing something?

Running nx project on angular 15

./node_modules/elastic-apm-node/lib/agent.js:242:12-55 - Warning: Critical dependency: the request of a dependency is an expression

./node_modules/elastic-apm-node/lib/config.js:1061:13-30 - Warning: Critical dependency: the request of a dependency is an expression

./node_modules/elastic-apm-node/lib/instrumentation/azure-functions.js:301:11-43 - Warning: Module not found: Error: Can't resolve '@azure/functions-core' in '/Users/rickvandermeij/Sources/demo/node_modules/elastic-apm-node/lib/instrumentation'

./node_modules/elastic-apm-node/lib/instrumentation/index.js:251:16-30 - Warning: Critical dependency: the request of a dependency is an expression

./node_modules/fast-json-stringify/index.js:14:11-33 - Warning: Module not found: Error: Can't resolve 'long' in '/Users/rickvandermeij/Sources/demo/node_modules/fast-json-stringify'

./node_modules/pino/lib/tools.js:170:26-62 - Warning: Module not found: Error: Can't resolve 'pino-pretty' in '/Users/rickvandermeij/Sources/demo/node_modules/pino/lib'



Compiled successfully.
{"log.level":"error","@timestamp":"2023-03-10T10:00:50.807Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error: error fetching APM Server version: connect ECONNREFUSED 127.0.0.1:8200"}

{"log.level":"error","@timestamp":"2023-03-10T10:00:50.808Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error (ECONNREFUSED): connect ECONNREFUSED 127.0.0.1:8200"}
{"log.level":"error","@timestamp":"2023-03-10T10:00:50.808Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error (ECONNREFUSED): connect ECONNREFUSED 127.0.0.1:8200"}

{"log.level":"info","@timestamp":"2023-03-10T10:00:50.825Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"x\"}"}

{"log.level":"error","@timestamp":"2023-03-10T10:00:50.911Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error (ECONNREFUSED): connect ECONNREFUSED 127.0.0.1:8200"}

A server error has occurred.
node exited with 1 code.
connect ECONNREFUSED ::1:64825

"server": {
	"executor": "@angular-builders/custom-webpack:server",
	"options": {
		"customWebpackConfig": {
			"path": "apps/content/extra-webpack.config.js",
			"mergeRules": {
				"module": {
					"rules": "append"
				}
			}
		},
		"outputPath": "dist/apps/content/server",
		"main": "apps/content/src/ssr.server.ts",
		"tsConfig": "apps/content/tsconfig.server.json"
	},