Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Work with optional dependencies #2

Conversation

RafalSumislawski
Copy link

Some libraries have option dependencies which they require in a try. For example mongodb requires mongodb-client-encryption this way here: https://github.com/mongodb/node-mongodb-native/blob/main/src/deps.ts#L277

With the current implementation of esbuild-node-plugin using mongodb leads to a bundling error:

✘ [ERROR] Cannot find module 'mongodb-client-encryption'
Require stack:
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/src/plugin.ts
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/test/test-app/build.ts
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/nodejs/esm-utils.js
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/mocha.js
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/cli/one-and-dones.js
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/cli/options.js
- /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/bin/mocha.js [plugin open-telemetry]

    node_modules/mongodb/lib/deps.js:117:42:
      117 │         mongodbClientEncryption = require('mongodb-client-encryption');
          ╵                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~

  This error came from the "onResolve" callback registered here:

    src/plugin.ts:16:0:
      16 │ 
         ╵ ^

    at setup (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/src/plugin.ts:16:4)
    at handlePlugins (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:1150:21)
    at buildOrContextImpl (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:873:5)
    at Object.buildOrContext (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:699:5)
    at /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:2023:15
    at new Promise (<anonymous>)
    at Object.build (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:2022:25)
    at build (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/node_modules/esbuild/lib/main.js:1873:51)
    at Object.<anonymous> (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/packages/esbuild-plugin-node/test/test-app/build.ts:20:21)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module.replacementCompile (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/append-transform/index.js:60:13)
    at Module.m._compile (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/ts-node/src/index.ts:1618:23)
    at Module.m._compile (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/ts-mocha/node_modules/ts-node/src/index.ts:439:23)
    at module.exports (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/default-require-extensions/js.js:7:9)
    at /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/ts-mocha/node_modules/ts-node/src/index.ts:442:12)
    at /Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/append-transform/index.js:64:4
    at require.extensions.<computed> (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/ts-node/src/index.ts:1621:12)
    at Object.<anonymous> (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/append-transform/index.js:64:4)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Function.Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.exports.requireOrImport (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/nodejs/esm-utils.js:53:16)
    at async Object.exports.loadFilesAsync (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/nodejs/esm-utils.js:100:20)
    at async singleRun (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/cli/run-helpers.js:162:3)
    at async Object.exports.handler (/Users/rsum/wrkspc/cx/opentelemetry-js-contrib-cx/node_modules/mocha/lib/cli/run.js:375:5)

Catching the error thrown in extractPackageAndModulePath and letting the blunder continue resolves the problem.

There is an issue about providing esbuild plugins with better ways to handle this evanw/esbuild#1127, but for now I don't see better alternatives.

Side note: this PR will make code using mongodb bundle successfully and work in runtime, but it doesn't make the mongodb instrumentation work. I'll submit a fix for in the next PR.

@github-actions github-actions bot requested a review from drewcorlin1 November 20, 2024 15:00
@RafalSumislawski RafalSumislawski changed the title Work with optional dependencies fix: Work with optional dependencies Nov 20, 2024
@drewcorlin1
Copy link
Owner

Thanks so much for this fix! Would you mind adding a test for a mongo span being emitted in packages/esbuild-plugin-node/test/plugin.test.ts?

@RafalSumislawski
Copy link
Author

I would love to do that, but it's problematic. Unlike redis, mongodb won't emit a span until we have successfully established a connection with the db. So either that test would need to be disabled by default like mongodb instrumentation tests are, which brings little to none value. Or it would need to spawn a mongodb (test-containers?), but I don't see any precedent of tests doing that (at least in this repo).

@drewcorlin1
Copy link
Owner

I would love to do that, but it's problematic. Unlike redis, mongodb won't emit a span until we have successfully established a connection with the db. So either that test would need to be disabled by default like mongodb instrumentation tests are, which brings little to none value. Or it would need to spawn a mongodb (test-containers?), but I don't see any precedent of tests doing that (at least in this repo).

Oh I see, didn't realize that. In my local fork of this I had been spinning up redis/localstack in containers that my CI job could talk to, but I also didn't see a precedent for that in the OTel repos. I can't think of a better solution than this.

What would you think of leaving a comment as something that would be nice to add in the future but we can merge as is?

@RafalSumislawski
Copy link
Author

Sure, I'll leave a comment.

BTW have you researched the CodeQL issue. I saw you commented about it in that PR. I think CodeQL has no supression mechanism, but maybe we can "fix" the issue. I mean apply some code change that would result in that no longer triggering.

@drewcorlin1
Copy link
Owner

I have not, if you're up for a way to get around it that'd be awesome, but also fine with me if you want to merge as is and I can handle it in the destination branch

Also add a TODO about mongodb test

move code around

move code around

Import instead of require

make prettier happy

prettier

test

remove commented code
@RafalSumislawski RafalSumislawski force-pushed the work-with-optional-dependencies branch from cc62cad to 715d18a Compare November 22, 2024 10:35
@RafalSumislawski
Copy link
Author

@drewcorlin1 I think this PR is ready

I was able to make CodeQL understand that we have rate limiting by chaining the register call after the fastify call.

@drewcorlin1
Copy link
Owner

@RafalSumislawski is this working when you run it locally? I'm trying to add a test using mongodb-memory-server but still seeing nothing being emitted.

/*
 * Copyright The OpenTelemetry Authors
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import { MongoClient } from 'mongodb';
import { MongoMemoryServer } from 'mongodb-memory-server';
import { buildTestSchema } from './graphql/schema';
import { createClient } from 'redis';
import fastify from 'fastify';
import { graphql } from './graphql/adapter';
import pino from 'pino';

const redisClient = createClient({ url: process.env.REDIS_URL });

export const server = fastify({
  logger: pino({}, pino.destination(1)),
  disableRequestLogging: true,
}).register(require('@fastify/rate-limit'), {
  // Use @fastify/rate-limit to avoid CodeQL "Missing rate limiting" error in CI
  global: true,
  max: 100,
  timeWindow: '1 minute',
});

server.get('/test', async req => {
  req.log.info({ hi: 'there' }, 'Log message from handler');

  await redisClient.get('key').catch(() => void 0);

  try {
    // TODO: fix
    // @ts-ignore
    const mongoDbUri = req.query?.mongoDbUri;
    if (!mongoDbUri) throw new Error('Missing mongoDbUri query param');
    const mongoClient = new MongoClient(mongoDbUri);
    const db = mongoClient.db('sample-database');
    const col = db.collection('sample-collection');
    const result = await col.findOne({ hello: 'test' });
    await mongoClient.close();
  } catch (e) {
    req.log.info(e, 'Error connecting to MongoDB');
  }

  return { hi: 'there' };
});

const schema = buildTestSchema();
const sourceList = `
  query {
    books {
      name
    }
  }
`;

server.get('/graphql', async req => {
  await graphql({ schema, source: sourceList });

  return { success: true };
});

server
  .listen({ port: 8080 })
  .then(async () => {
    const mongod = await MongoMemoryServer.create();
    const mongoDbUri = mongod.getUri();
    try {
      await server
        .inject()
        .get('/test')
        .query({ mongoDbUri })
        .end()
        .catch(err => {
          throw err;
        });
      await server
        .inject()
        .get('/graphql')
        .end()
        .catch(err => {
          throw err;
        });
    } finally {
      await mongod.stop().catch(() => void 0);
      return server.close();
    }
  })
  .catch(err => {
    throw err;
  });

@RafalSumislawski
Copy link
Author

@drewcorlin1

As I mentioned in the PR's description, this PR just makes it not crash in bundle-time. For the instrumentation to actually work, you also need #3

@drewcorlin1 drewcorlin1 merged commit f1a5fdf into drewcorlin1:esbuild-plugin Dec 3, 2024
17 checks passed
@drewcorlin1
Copy link
Owner

Missed that it wasn't rebased, thanks for this! pushed a test for the mongoDB test to the esbuild-plugin branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants