Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recent fast-xml-parser vulnerability now shows dozens of high severity vulnerabilities during npm install #6331

Closed
3 tasks done
purplepenguin42 opened this issue Jul 29, 2024 · 33 comments
Labels
bug This issue is a bug. p1 This is a high priority issue

Comments

@purplepenguin42
Copy link

Checkboxes for prior research

Describe the bug

Related:

When running an npm install with latest versions of things like @aws-sdk/client-dynamodb, @aws-sdk/client-kms, @aws-sdk/client-lambda, etc. you are now greeted with a bold red text about 18 high severity vulnerabilities. I'm using the latest (i.e. published a few hours ago versions), and npm is still showing the warnings.

Is there a timeline when the rest of the ecosystem will be updated? Previously (e.g. yesterday), the same install resulted in 0 warnings.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v22.5.1

Reproduction Steps

Run npm install with any of the modular SDK packages on the latest version.

Observed Behavior

A bunch of high severity vulnerability warnings.

Expected Behavior

No warnings.

Possible Solution

No response

Additional Information/Context

No response

@purplepenguin42 purplepenguin42 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 29, 2024
@purplepenguin42
Copy link
Author

Ignore the exact number (18), it'll vary depending on which modules you're installing.

$ npm install @aws-sdk/[email protected]

added 90 packages, and audited 91 packages in 1s

2 packages are looking for funding
  run `npm fund` for details

13 high severity vulnerabilities

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

@purplepenguin42 purplepenguin42 changed the title Recent fast-xml-parser vulnerability now shows 18 high severity vulnerabilities during npm install Recent fast-xml-parser vulnerability now shows dozens of high severity vulnerabilities during npm install Jul 29, 2024
@kiarash-malekzadeh
Copy link

kiarash-malekzadeh commented Jul 30, 2024

Having the same issue with our:

  • @aws-sdk/client-s3
  • @aws-sdk/client-sts

Updating to latest versions did not help since the fast-xml-parser version is still 4.2.5 which needs to be upgraded to 4.4.1

@seriousme
Copy link

See #6330

@rlsf
Copy link

rlsf commented Jul 30, 2024

@seriousme is there a timeline for a release of fixed version?

@seriousme
Copy link

I don't know, I don't work for Amazon.
I only found the PR and I thought it might help linking it here.

@bhaveshj459
Copy link

@kiarash-malekzadeh have you found any sol for @AWS-SDK and all

@rlsf
Copy link

rlsf commented Jul 30, 2024

@trivikr is there a timeline for a release of fixed version?

@FarhanHossain00
Copy link

I'm also having a dozen of vulnerabilities'
Will install @aws-sdk/[email protected], which is a breaking change
node_modules/fast-xml-parser
@aws-sdk/core >=3.529.1
Depends on vulnerable versions of fast-xml-parser
node_modules/@aws-sdk/core
@aws-sdk/client-s3 >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sso-oidc
Depends on vulnerable versions of @aws-sdk/client-sts
Depends on vulnerable versions of @aws-sdk/core
Depends on vulnerable versions of @aws-sdk/credential-provider-node
node_modules/@aws-sdk/client-s3
@aws-sdk/client-sso >=3.529.1
Depends on vulnerable versions of @aws-sdk/core
node_modules/@aws-sdk/client-sso
@aws-sdk/credential-provider-sso >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sso
Depends on vulnerable versions of @aws-sdk/token-providers
node_modules/@aws-sdk/credential-provider-sso
@aws-sdk/credential-provider-node >=3.529.1
Depends on vulnerable versions of @aws-sdk/credential-provider-ini
Depends on vulnerable versions of @aws-sdk/credential-provider-sso
Depends on vulnerable versions of @aws-sdk/credential-provider-web-identity
node_modules/@aws-sdk/credential-provider-node
@aws-sdk/client-sso-oidc >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sts
Depends on vulnerable versions of @aws-sdk/core
Depends on vulnerable versions of @aws-sdk/credential-provider-node
node_modules/@aws-sdk/client-sso-oidc
@aws-sdk/client-sts >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sso-oidc
Depends on vulnerable versions of @aws-sdk/core
Depends on vulnerable versions of @aws-sdk/credential-provider-node
node_modules/@aws-sdk/client-sts
@aws-sdk/credential-provider-ini >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sts
Depends on vulnerable versions of @aws-sdk/credential-provider-sso
Depends on vulnerable versions of @aws-sdk/credential-provider-web-identity
node_modules/@aws-sdk/credential-provider-ini
@aws-sdk/credential-provider-web-identity >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sts
node_modules/@aws-sdk/credential-provider-web-identity
@aws-sdk/token-providers >=3.529.1
Depends on vulnerable versions of @aws-sdk/client-sso-oidc
node_modules/@aws-sdk/token-providers

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

is there a timeline for a release of fixed version?

The new version https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.621.0 with updated version of fast-xml-parser is expected to release around 12:30 PM Pacific today (Tue, July 30th)

@xfournet
Copy link

xfournet commented Jul 30, 2024

@trivikr thanks! Quick question, why do you pin the dependencies version in @aws-sdk packages ? It prevent to fix the issue by ourself.

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

why do you pin the dependencies version in @aws-sdk packages ?

We pin third-party dependencies in @aws-sdk packages, so that your AWS applications do not break by some bad version update in a package that we don't maintain.


Imagine a scenario where fast-xml-parser mistakenly releases a bad update which breaks the way XML is parsed and SDK uses an unpinned version. It'll break all XML protocol based clients in applications which do no use lockfiles as soon as newer version of the dependency is installed.

This issue is difficult to triage since it happens when bad version is released in a third-party dependency. After triaging, the immediate fix also requires a code change in application, like pinning the third-party dependency version in package.json overrides. This fix is expensive as it requires commit+review+build+deploy in application code and we also need to educate the application maintainers.

The most popular XML based protocol client, i.e. @aws-sdk/client-s3, has 6M+ downloads per week. So you can imagine how bad that can be.


On other hand, when there's "security vulnerability" in XML parsing like this case, your AWS applications are actually NOT impacted. This is because the SDK parses XML content only from trusted sources, i.e. responses from AWS servers. These responses are also obtained over HTTPS by default, so there's no possibility of PITM attack to change the response.

We recognize that these security advisories nonetheless cause build failures and auditing issues, and we provide a timely response. Your AWS applications are still safe with older versions of fast-xml-parser with vulnerability in XML parsing. But if you or your team is worried about vulnerability report, just update to the latest version.

New latest version usually gets published within 24 hours, like https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.621.0 will publish around 12:30 PM Pacific today. But we can also also publish it out of window, if there's a real issue in the SDK.

@trivikr trivikr added p1 This is a high priority issue queued This issues is on the AWS team's backlog and removed needs-triage This issue or PR still needs to be triaged. labels Jul 30, 2024
@surecloud-Awalia
Copy link

surecloud-Awalia commented Jul 30, 2024

Do you know if this fix will also be released simultaneously for sdk v2 ?

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

Do you know if this fix also be released simultaneously for sdk v2 ?

The AWS SDK for JavaScript (v2) does NOT have a direct/transient dependency on fast-xml-parser

It can be verified by installing the SDK in a new folder as follows:

$ test-deps> npm init -y

$ test-deps> npm install aws-sdk

$ test-deps> npm why aws-sdk
[email protected]
node_modules/aws-sdk
  aws-sdk@"^2.1664.0" from the root project

$ test-deps> npm why fast-xml-parser

@Ardakilic
Copy link

We encountered this on our project today.

Somehow 3.620.1 did not fix for us, so we had to use overrides section temporarily on our package.json:

"overrides": {
    "@aws-sdk/client-s3": {
        "fast-xml-parser": "4.4.1",
        "@aws-sdk/client-sts": {
            "fast-xml-parser": "4.4.1"
        }
    }
},

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

Somehow 3.620.1 did not fix for us

The fix is going to be published in v3.621.0

Details in prior comment #6331 (comment)

@kuhe kuhe added pending-release This issue will be fixed by an approved PR that hasn't been released yet. and removed queued This issues is on the AWS team's backlog labels Jul 30, 2024
@purplepenguin42
Copy link
Author

purplepenguin42 commented Jul 30, 2024

@trivikr

We pin third-party dependencies in @aws-sdk packages, so that your AWS applications do not break by some bad version update in a package that we don't maintain.

This isn't how pinning works, mainly because you can't actually pin a dependency tree. You're pinning the top-level dependencies, but all of the transitive dependencies still freely update based on semver ranges. The reality is the current design is basically the worst of both worlds, you still aren't pinning most of the SDK's actual dependencies but you are pinning a handful, which makes it impossible for these issues to simply be non-issues by letting consumers run a fresh install.

Put another way, the current approach is like closing one window in a house, while leaving all the other windows and doors open, and then thinking you're safer because you closed that one window, when in reality it has no real effect on anything.

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

mainly because you can't actually pin a dependency tree

@purplepenguin42 I tried my best to explain the issue in a concise way in the comment #6331 (comment) providing specific examples on how failures can happen in both cases, and the cost of fixing them. I also didn't use some any analogy to put it in another way.

Can you create a feature request and provide a minimal repro on how a bad update in third-party dependency, like fast-xml-parser, will not immediately break applications without lockfiles as soon as they run new install step. I can go through your example, and provide a counter example, if needed.

That way, we keep this issue specific to [email protected] update, and not digress.

@trivikr trivikr pinned this issue Jul 30, 2024
@purplepenguin42
Copy link
Author

@trivikr

What I'm trying to get across is you aren't doing what you actually think you are doing. Yes, you can pin fast-xml-parser to a specific version, but that does absolutely nothing for all the rest of the SDK's dependencies that aren't top-level dependencies.

For example, fast-xml-parser has a dependency on strnum. Despite you pinning fast-xml-parser, strnum will still pull in the latest updates regardless. If strnum releases a bad update, guess what? You're still in the exact same boat.

The point I'm trying to make is pinning fast-xml-parser only affects bad updates specifically in fast-xml-parser (or whatever other top-level dependencies), it has no effect whatsoever on transitive dependencies, which make up the bulk of the SDK's total footprint. You can only pin an individual package declared in your own package.json, all dependencies under that still freely float and pull whatever version satisfies semver. You fundamentally cannot pin a full dependency tree, but you seem to believe the current design is doing just that, which is simply incorrect and potentially dangerous.

tl;dr - you pinning a handful of dependencies does more harm than good; you are only preventing "bad updates" to a tiny percentage of the SDK's overall dependencies, while at the same time preventing people from easily using the latest versions, and instead forcing users to rely on manual releases from the SDK in situations like this.

Regardless, I won't comment any further on this thread. Feel free to make another tracking issue, but it doesn't appear the SDK team actually understands how their own system currently works, which is a bit scary.

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

The @aws-sdk/[email protected] was published at 12:19 PM Pacific

$ npm view @aws-sdk/client-s3 time --json | grep 3.621.0
  "3.621.0": "2024-07-30T19:19:32.730Z"

I verified that it includes [email protected] as follows:

$ test-deps> npm init -y

$ test-deps> npm install @aws-sdk/[email protected] --save-exact

$ test-deps> npm why fast-xml-parser | grep fast-xml-parser
[email protected]
node_modules/fast-xml-parser
  fast-xml-parser@"4.4.1" from @aws-sdk/[email protected]

If you're blocked on this vulnerability report, please upgrade your SDK version to v3.621.0

@trivikr trivikr closed this as completed Jul 30, 2024
@trivikr trivikr removed the pending-release This issue will be fixed by an approved PR that hasn't been released yet. label Jul 30, 2024
@Ardakilic
Copy link

Hello, can confirm 3.621.0 fixes the vulnerability on my dependency tree. Thank you!

@kiarash-malekzadeh
Copy link

kiarash-malekzadeh commented Jul 30, 2024

@trivikr Still having the 4.2.5 dependency under this in package-lock.json:

"node_modules/@aws-sdk/client-sts": {
           "version": "3.454.0",
           "dependencies": {
               "fast-xml-parser": "4.2.5",
           },
       } 

Is there anything else that should be updated besides @aws-sdk/client-s3 and aws-sdk? Please note that we have not installed the @aws-sdk/client-sts and it does not exist in package.json.

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

Is there anything else that should be updated besides @aws-sdk/client-s3

Update all your @aws-sdk/* clients to v3.621.0.

You've only provided a subset of your lockfile contains @aws-sdk/[email protected], which is likely added as a transitive dependency from some other v3.454.0 client.

@kiarash-malekzadeh
Copy link

@trivikr We have only the following installed:

  • "@aws-sdk/client-s3": "^3.621.0",
  • "aws-sdk": "^2.1665.0",

I think the dependencies of "aws-sdk" have not been updated.

@kiarash-malekzadeh
Copy link

@trivikr This is what I get from npm audit:

fast-xml-parser  <4.4.1
Severity: high
fast-xml-parser vulnerable to ReDOS at currency parsing - https://github.com/advisories/GHSA-mpg4-rc92-vx8v
fix available via `npm audit fix`
node_modules/fast-xml-parser
  @aws-sdk/client-sts  <=3.525.0
  Depends on vulnerable versions of fast-xml-parser
  node_modules/@aws-sdk/client-sts
    @aws-sdk/client-cognito-identity  3.12.0 - 3.525.0
    Depends on vulnerable versions of @aws-sdk/client-sts
    node_modules/@aws-sdk/client-cognito-identity
      @aws-sdk/credential-provider-cognito-identity  3.12.0 - 3.525.0
      Depends on vulnerable versions of @aws-sdk/client-cognito-identity
      node_modules/@aws-sdk/credential-provider-cognito-identity
    @aws-sdk/credential-providers  <=3.525.0
    Depends on vulnerable versions of @aws-sdk/client-cognito-identity
    Depends on vulnerable versions of @aws-sdk/client-sts
    Depends on vulnerable versions of @aws-sdk/credential-provider-cognito-identity
    node_modules/@aws-sdk/credential-providers

ip  *
Severity: high

@trivikr
Copy link
Member

trivikr commented Jul 30, 2024

@kiarash-malekzadeh You can examine your lockfile by running npm why @aws-sdk/client-sts, which will provide information on why the dependency is imported. And update the direct dependency accordingly.

From the logs, it looks like you have the following direct dependency

  "@aws-sdk/client-sts": "<=3.525.0",

@kiarash-malekzadeh
Copy link

@trivikr Thanks for the help. Fixed the issue.

@bhaveshj459
Copy link

bhaveshj459 commented Jul 31, 2024

@trivikr this is showing in my audit files

fast-xml-parser  <4.4.1
Severity: high
fast-xml-parser vulnerable to ReDOS at currency parsing - https://github.com/advisories/GHSA-mpg4-rc92-vx8v
fix available via `npm audit fix --force`
Will install [email protected], which is a breaking change
node_modules/@aws-sdk/client-sts/node_modules/fast-xml-parser
  @aws-sdk/client-sts  <=3.525.0
  Depends on vulnerable versions of fast-xml-parser
  node_modules/@aws-sdk/client-sts
    @aws-sdk/client-firehose  3.12.0 - 3.525.0
    Depends on vulnerable versions of @aws-sdk/client-sts
    node_modules/@aws-sdk/client-firehose
      @aws-amplify/analytics  >=7.0.0
      Depends on vulnerable versions of @aws-sdk/client-firehose
      Depends on vulnerable versions of @aws-sdk/client-kinesis
      Depends on vulnerable versions of @aws-sdk/client-personalize-events
      node_modules/@aws-amplify/analytics
        aws-amplify  >=6.0.0
        Depends on vulnerable versions of @aws-amplify/analytics
        node_modules/aws-amplify
          @aws-amplify/ui  <=0.0.0-theming-v2-f8422a3-20240627193426 || >=6.0.0
          Depends on vulnerable versions of aws-amplify
          node_modules/@aws-amplify/ui
          @aws-amplify/ui-angular  <=0.0.0-theming-v2-f8422a3-20240627193426 || >=5.0.0
          Depends on vulnerable versions of @aws-amplify/ui
          Depends on vulnerable versions of aws-amplify
          node_modules/@aws-amplify/ui-angular
    @aws-sdk/client-kinesis  3.12.0 - 3.525.0
    Depends on vulnerable versions of @aws-sdk/client-sts
    node_modules/@aws-sdk/client-kinesis
    @aws-sdk/client-personalize-events  3.12.0 - 3.525.0
    Depends on vulnerable versions of @aws-sdk/client-sts
    node_modules/@aws-sdk/client-personalize-events

@trivikr
Copy link
Member

trivikr commented Jul 31, 2024

@bhaveshj459 The answer is similar to one given in previous comment, and your console also shows @aws-sdk/client-sts <=3.525.0

#6331 (comment)

@bhaveshj459
Copy link

@trivikr I have tried it but its not working for me also I have made an new angular application to make sure other deps are not causing this issue but found no difference.

@jlove-dev
Copy link

I'm getting the same thing. npm audit:

# npm audit report

fast-xml-parser  <4.4.1
Severity: high
fast-xml-parser vulnerable to ReDOS at currency parsing - https://github.com/advisories/GHSA-mpg4-rc92-vx8v
fix available via `npm audit fix --force`
Will install [email protected], which is a breaking change
node_modules/fast-xml-parser
  @aws-sdk/client-sts  <=3.525.0 || 3.529.1 - 3.620.1
  Depends on vulnerable versions of @aws-sdk/client-sso-oidc
  Depends on vulnerable versions of @aws-sdk/core
  Depends on vulnerable versions of @aws-sdk/credential-provider-node
  Depends on vulnerable versions of fast-xml-parser
  node_modules/@aws-sdk/client-cognito-identity-provider/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-firehose/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-kinesis/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-personalize-events/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-s3/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-sso-oidc/node_modules/@aws-sdk/client-sts
  node_modules/@aws-sdk/client-sts
    @aws-sdk/client-cognito-identity  3.529.1 - 3.620.1
    Depends on vulnerable versions of @aws-sdk/client-sso-oidc
    Depends on vulnerable versions of @aws-sdk/client-sts
    Depends on vulnerable versions of @aws-sdk/core
    Depends on vulnerable versions of @aws-sdk/credential-provider-node
    node_modules/@aws-sdk/client-cognito-identity
      @aws-sdk/credential-provider-cognito-identity  3.529.1 - 3.620.1
      Depends on vulnerable versions of @aws-sdk/client-cognito-identity

npm why:

npm why @aws-sdk/client-sts
@aws-sdk/[email protected]
node_modules/@aws-amplify/analytics/node_modules/@aws-sdk/client-sts
  @aws-sdk/client-sts@"3.398.0" from @aws-sdk/[email protected]
  node_modules/@aws-amplify/analytics/node_modules/@aws-sdk/client-firehose
    @aws-sdk/client-firehose@"3.398.0" from @aws-amplify/[email protected]
    node_modules/@aws-amplify/analytics
      @aws-amplify/analytics@"7.0.40" from [email protected]
      node_modules/aws-amplify
        aws-amplify@"^6.3.8" from the root project
  @aws-sdk/client-sts@"3.398.0" from @aws-sdk/[email protected]
  node_modules/@aws-amplify/analytics/node_modules/@aws-sdk/client-kinesis
    @aws-sdk/client-kinesis@"3.398.0" from @aws-amplify/[email protected]
    node_modules/@aws-amplify/analytics
      @aws-amplify/analytics@"7.0.40" from [email protected]
      node_modules/aws-amplify
        aws-amplify@"^6.3.8" from the root project
  @aws-sdk/client-sts@"3.398.0" from @aws-sdk/[email protected]
  node_modules/@aws-amplify/analytics/node_modules/@aws-sdk/client-personalize-events
    @aws-sdk/client-personalize-events@"3.398.0" from @aws-amplify/[email protected]
    node_modules/@aws-amplify/analytics
      @aws-amplify/analytics@"7.0.40" from [email protected]
      node_modules/aws-amplify
        aws-amplify@"^6.3.8" from the root project

I've tried resolving this but my dependencies show:

"@aws-sdk/client-sts": "^3.606.0"
"aws-amplify": "^6.3.8"

@trivikr
Copy link
Member

trivikr commented Aug 1, 2024

@jlove-dev The AWS SDK in your application is imported from Amplify.

Please refer to aws-amplify/amplify-js#13670, for updates in @aws-amplify/*

@jlove-dev
Copy link

Thank you @trivikr that really clears things up!

Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. p1 This is a high priority issue
Projects
None yet
Development

No branches or pull requests