New check: Check for vulnerabilities in your dependency trees

[inferno-chromium]

Is your feature request related to a problem? Please describe.
Right now, vulnerabilities check gives vulns in the package itself, but does not say anything about known vulns in its deps tree.

Describe the solution you'd like
Either use Open Source Insights api or something similar, to create a new check DepsVulns and provide information on known vuln in the deps.

Describe alternatives you've considered
NA

Additional context
NA

[laurentsimon]

cc @naveensrinivasan

[naveensrinivasan]

I agree. If we are going down the rabbit hole of the dependency tree why just look into Vulns alone. What about the health of the transitive dependency tree?

We already have this Scorecard data in our BigQuery. The steps involved are dependency tree and getting scorecard data from BigQuery for each of the dependencies.

shameless plug I built this to identify the scorecard for all the go dependencies https://github.com/naveensrinivasan/scorecarddata

[inferno-chromium]

@naveensrinivasan - the problem is we need something to work across languages. go is easy, we need to support atleast a few more ecosystems.

[laurentsimon]

Note that BQ table contains the scorecard's repo data, but the missing link is package version/hash. I don't think we can use BQ table because of that, and the data must come from another source. We can potentially query each ecosystem's database ourselves, but this still requires us to parse the manifest file...

If OSI API is available, that's the way to go. Otherwise we should consider creating a service that runs existing ecosystem's tools like pip-audit, cargo-audit, and expose the results via an API.

[naveensrinivasan]

@naveensrinivasan - the problem is we need something to work across languages. go is easy, we need to support atleast a few more ecosystems.

I agree. But I think we should scope it as not just Vulns that was the point.

[inferno-chromium]

@naveensrinivasan - the problem is we need something to work across languages. go is easy, we need to support atleast a few more ecosystems.

I agree. But I think we should scope it as not just Vulns that was the point.

ah that, that i totally agree, it could be even aggregate scorecard calc on deps too instead of just vulns

[azeemshaikh38]

I agree with Naveen. I think finding transitive dependencies (and scoring them) shouldn't be a Scorecard check. Instead, I think we should aim to partner with someone like Open Source Insights or an SBOM generation tool and plug our Scorecard result in their output dependency tree. The biggest advantage being, this will allow a tool like AllStar to apply fine-grained and complex policies on not just the repo itself but also on its dependencies.

[naveensrinivasan]

I think finding transitive dependencies (and scoring them) shouldn't be a Scorecard check. Instead, I think we should aim to partner with someone like Open Source Insights or an SBOM generation tool and plug our Scorecard result in their output dependency tree.

• I have a different opinion if we don't do it, which project is going to do it? Unless we decide to build a project that has that information.

• Another blocker for other projects (like SBOM generation) is that they would have to have an auth for BigQuery because Scorecard doesn't expose this data via an HTTP API. Hypothetically if an SBOM tool would bring in this feature then the end-users would have to provide auth for BigQuery which is a blocker for most of them.

The biggest advantage being, this will allow a tool like AllStar to apply fine-grained and complex policies on not just the repo itself but also on its dependencies.

AllStar is specific to GitHub and Scorecard would eventually not be specific to GitHub. AllStar is a not CLI tool that everyone can consume.

[laurentsimon]

another angle for this is that scorecard has complementary information to OSI's data: we look at workflows, shell scripts, makefiles (soon), GCB config (soon), so we could also build and expose this data "dependencies in CICD", which is useful during incidence response and that nobody is really looking into atm. I suppose this would even be useful in the context of criticality score.

[naveensrinivasan]

another angle for this is that scorecard has complementary information to OSI's data: we look at workflows, shell scripts, makefiles (soon), GCB config (soon), so we could also build and expose this data "dependencies in CICD", which is useful during incidence response and that nobody is really looking into atm. I suppose this would even be useful in the context of criticality score.

So are you in favor of having the check as part of scorecard?

[naveensrinivasan]

another angle for this is that scorecard has complementary information to OSI's data: we look at workflows, shell scripts, makefiles (soon), GCB config (soon), so we could also build and expose this data "dependencies in CICD", which is useful during incidence response and that nobody is really looking into atm. I suppose this would even be useful in the context of criticality score.

I agree! 👍

[laurentsimon]

related https://blog.trailofbits.com/2021/12/16/it-depends/
wonder if other tools like dependabot/renovatebot provide an API to parse and output vulnerabilities from the various ecosystems

cc @rarkins

[naveensrinivasan]

I had a quick look at the repo for parsing go. It uses string parsing which is brittle and likely to break. I am not excited about that project.

[laurentsimon]

another angle for this is that scorecard has complementary information to OSI's data: we look at workflows, shell scripts, makefiles (soon), GCB config (soon), so we could also build and expose this data "dependencies in CICD", which is useful during incidence response and that nobody is really looking into atm. I suppose this would even be useful in the context of criticality score.

I agree! 👍

@oliverchang @calebbrown

[seelder]

@spencerschrock - I think you said that the original "vulnerabilities" check has been addressed via the use of OSV-scanner? In which case this might be closed (with aggregation for additional checks being a separate issue)?

I figured out that the issues I was seeing were likely in OSV-Scanner (416 and 332). Edit: (These issues were the reason the check didn't seem to be working in a very specific example I was looking at)

Edit2: If it needs to remain open, this is probably something I can/should help with (depending on why it needs to remain open)

[spencerschrock]

@spencerschrock - I think you said that the original "vulnerabilities" check has been addressed via the use of OSV-scanner? In which case this might be closed (with aggregation for additional checks being a separate issue)?

Agreed. Sorry for the delayed response.