Add Snyk integration (#2070)

Closes #365 Signed-off-by: Apurva <[email protected]> Signed-off-by: Sahiba Mittal <[email protected]> Signed-off-by: mehab <[email protected]> Co-authored-by: Apurva <[email protected]> Co-authored-by: mehab <[email protected]> Co-authored-by: japurva1502 <[email protected]>
DependencyTrack · Oct 21, 2022 · 7878b03 · 7878b03
2 parents e5aa276 + 1d5fcdc
commit 7878b03
Show file tree

Hide file tree

Showing 26 changed files with 1,132 additions and 31 deletions.
diff --git a/docs/_docs/analysis-types/known-vulnerabilities.md b/docs/_docs/analysis-types/known-vulnerabilities.md
@@ -22,6 +22,9 @@ The internal analyzer relies on a dictionary of vulnerable software. This dictio
 NVD, GitHub Advisories, or VulnDB mirroring is performed. The internal analyzer is applicable to all components with valid 
 CPEs, including application, operating system, and hardware components, and all components with Package URLs.
 
+**NOTE**: Currently, vulnerable software describing affected package is treated as 'append-only' meaning there might be some entries no longer reported by the vulnerability source.
+Fix is under progress and can be tracked via issue [#1815](https://github.com/DependencyTrack/dependency-track/issues/1815).
+
 ### OSS Index Analyzer
 
 OSS Index is a service provided by Sonatype which identifies vulnerabilities in third-party components. The service 

diff --git a/docs/_docs/datasources/internal-components.md b/docs/_docs/datasources/internal-components.md
@@ -2,7 +2,7 @@
 title: Internal Components
 category: Datasources
 chapter: 4
-order: 8
+order: 9
 ---
 
 Organizations have the ability to specify a namespace and/or name which represents internally

diff --git a/docs/_docs/datasources/private-vuln-repo.md b/docs/_docs/datasources/private-vuln-repo.md
@@ -2,7 +2,7 @@
 title: Private Vulnerability Repository
 category: Datasources
 chapter: 4
-order: 9
+order: 10
 redirect_from:
   - /usage/private-vuln-repo/
 ---

diff --git a/docs/_docs/datasources/repositories.md b/docs/_docs/datasources/repositories.md
@@ -2,7 +2,7 @@
 title: Repositories
 category: Datasources
 chapter: 4
-order: 7
+order: 8
 ---
 
 Dependency-Track relies on integration with repositories to help identify metadata that may be useful

diff --git a/docs/_docs/datasources/routing.md b/docs/_docs/datasources/routing.md
@@ -2,7 +2,7 @@
 title: Datasource Routing
 category: Datasources
 chapter: 4
-order: 6
+order: 7
 ---
 
 Components often belong to one or more ecosystems. These ecosystems typically have one or more sources of 

diff --git a/docs/_docs/datasources/snyk.md b/docs/_docs/datasources/snyk.md
@@ -0,0 +1,41 @@
+---
+title: Snyk
+category: Datasources
+chapter: 4
+order: 5
+---
+
+[Snyk](https://security.snyk.io) is a platform allowing you to scan, prioritize, and fix security vulnerabilities in your own code, open source dependencies, container images, and Infrastructure as Code (IaC) configurations.
+
+It is a developer security platform. Integrating directly into development tools, workflows, and automation pipelines, Snyk makes it easy for teams to find, prioritize, and fix security vulnerabilities in code, dependencies, containers, and infrastructure as code. Supported by industry-leading application and security intelligence, Snyk puts security expertise in any developer's toolkit.
+
+Dependency-Track integrates with Snyk using its [REST API](https://apidocs.snyk.io/). Dependency-Track does not mirror Snyk entirely,
+but it does consume vulnerabilities on a 'as-identified' basis.
+
+The Snyk integration is disabled by default.
+
+### Authentication
+
+User must get API token from Snyk. You can find your token in your [General Account Settings](https://snyk.io/account/) after you register with Snyk and log in. See [Authentication for API](https://docs.snyk.io/snyk-api-info/authentication-for-api).
+
+Provide the token (**without** 'token' prefixed) in the configuration as shown below.
+
+### Configuration
+
+**Organization ID** can be set at in the [Settings](https://docs.snyk.io/products/snyk-code/cli-for-snyk-code/before-you-start-set-the-organization-for-the-cli-tests/finding-the-snyk-id-and-internal-name-of-an-organization) page of the Organization on the Web UI.
+
+**Snyk base URL** is set by default, can be changed per requirement.
+
+**Snyk API version** is set by default to latest version. It is updated every 6 months and might get expired causing API communication failure in which case it will be updated in next upcoming DT release.
+User can change it manually here. Please refer [API](https://apidocs.snyk.io/?version=2022-10-06#overview) to submit the correct version.
+**Number of threads for Snyk Analyzer to use** Snyk analyzer is implemented with multithreading model to complete the analysis faster. The number of threads that would be used is configurable. By default, it is set to 10. The value can be overridden by exporting this environment variable: `SNYK_THREAD_BATCH_SIZE`. The value can be set based on the configuration of the machine. 
+
+![](../../images/snyk-configuration.png)
+
+### Understanding Snyk's CVSS analysis
+
+The majority of vulnerabilities published by Snyk originate from proprietary research, public information sources, or through 3rd party disclosures.
+
+When evaluating the severity of a vulnerability, it's important to note that there is no single CVSS vector - there are multiple CVSS vectors defined by multiple vendors, with the National Vulnerability Database (NVD) being one of them.
+
+**NOTE:** For Beta version, user can select either from NVD or SNYK to prioritize the cvss vectors.
diff --git a/docs/_docs/datasources/vulndb.md b/docs/_docs/datasources/vulndb.md
@@ -2,7 +2,7 @@
 title: VulnDB
 category: Datasources
 chapter: 4
-order: 5
+order: 6
 ---
 
 VulnDB, a subscription service offered by Risk Based Security, offers a comprehensive and continuously updated 

diff --git a/docs/images/snyk-configuration.png b/docs/images/snyk-configuration.png
diff --git a/src/main/java/org/dependencytrack/common/ConfigKey.java b/src/main/java/org/dependencytrack/common/ConfigKey.java
@@ -0,0 +1,26 @@
+package org.dependencytrack.common;
+
+import alpine.Config;
+
+public enum ConfigKey implements Config.Key{
+    SNYK_THREAD_BATCH_SIZE("snyk.thread.batch.size", 10);
+
+    private final String propertyName;
+    private final Object defaultValue;
+
+    ConfigKey(final String propertyName, final Object defaultValue) {
+        this.propertyName = propertyName;
+        this.defaultValue = defaultValue;
+    }
+
+    @Override
+    public String getPropertyName() {
+        return propertyName;
+    }
+
+    @Override
+    public Object getDefaultValue() {
+        return defaultValue;
+    }
+
+}
diff --git a/src/main/java/org/dependencytrack/event/EventSubsystemInitializer.java b/src/main/java/org/dependencytrack/event/EventSubsystemInitializer.java
@@ -49,6 +49,7 @@
 import org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask;
 import org.dependencytrack.tasks.scanners.InternalAnalysisTask;
 import org.dependencytrack.tasks.scanners.OssIndexAnalysisTask;
+import org.dependencytrack.tasks.scanners.SnykAnalysisTask;
 import org.dependencytrack.tasks.scanners.VulnDbAnalysisTask;
 
 import javax.servlet.ServletContextEvent;
@@ -91,6 +92,7 @@ public void contextInitialized(final ServletContextEvent event) {
         EVENT_SERVICE.subscribe(VulnDbAnalysisEvent.class, VulnDbAnalysisTask.class);
         EVENT_SERVICE.subscribe(VulnerabilityAnalysisEvent.class, VulnerabilityAnalysisTask.class);
         EVENT_SERVICE.subscribe(PortfolioVulnerabilityAnalysisEvent.class, VulnerabilityAnalysisTask.class);
+        EVENT_SERVICE.subscribe(SnykAnalysisEvent.class, SnykAnalysisTask.class);
         EVENT_SERVICE.subscribe(RepositoryMetaEvent.class, RepositoryMetaAnalyzerTask.class);
         EVENT_SERVICE.subscribe(ComponentMetricsUpdateEvent.class, ComponentMetricsUpdateTask.class);
         EVENT_SERVICE.subscribe(ProjectMetricsUpdateEvent.class, ProjectMetricsUpdateTask.class);
@@ -135,6 +137,7 @@ public void contextDestroyed(final ServletContextEvent event) {
         EVENT_SERVICE.unsubscribe(ProjectMetricsUpdateTask.class);
         EVENT_SERVICE.unsubscribe(PortfolioMetricsUpdateTask.class);
         EVENT_SERVICE.unsubscribe(VulnerabilityMetricsUpdateTask.class);
+        EVENT_SERVICE.unsubscribe(SnykAnalysisTask.class);
         EVENT_SERVICE.unsubscribe(CloneProjectTask.class);
         EVENT_SERVICE.unsubscribe(FortifySscUploadTask.class);
         EVENT_SERVICE.unsubscribe(DefectDojoUploadTask.class);

diff --git a/src/main/java/org/dependencytrack/event/SnykAnalysisEvent.java b/src/main/java/org/dependencytrack/event/SnykAnalysisEvent.java
@@ -0,0 +1,22 @@
+package org.dependencytrack.event;
+
+import org.dependencytrack.model.Component;
+
+import java.util.List;
+
+/**
+ * Defines an event used to start an analysis via Snyk REST API.
+ */
+public class SnykAnalysisEvent extends VulnerabilityAnalysisEvent {
+
+    public SnykAnalysisEvent() { }
+
+    public SnykAnalysisEvent(final Component component) {
+        super(component);
+    }
+
+    public SnykAnalysisEvent(final List<Component> components) {
+        super(components);
+    }
+
+}
diff --git a/src/main/java/org/dependencytrack/model/ConfigPropertyConstants.java b/src/main/java/org/dependencytrack/model/ConfigPropertyConstants.java
@@ -46,7 +46,13 @@ public enum ConfigPropertyConstants {
     SCANNER_VULNDB_ENABLED("scanner", "vulndb.enabled", "false", PropertyType.BOOLEAN, "Flag to enable/disable VulnDB"),
     SCANNER_VULNDB_OAUTH1_CONSUMER_KEY("scanner", "vulndb.api.oauth1.consumerKey", null, PropertyType.STRING, "The OAuth 1.0a consumer key"),
     SCANNER_VULNDB_OAUTH1_CONSUMER_SECRET("scanner", "vulndb.api.oath1.consumerSecret", null, PropertyType.ENCRYPTEDSTRING, "The OAuth 1.0a consumer secret"),
-    SCANNER_ANALYSIS_CACHE_VALIDITY_PERIOD("scanner", "analysis.cache.validity.period","864000", PropertyType.NUMBER, "Validity period for individual component analysis cache"),
+    SCANNER_ANALYSIS_CACHE_VALIDITY_PERIOD("scanner", "analysis.cache.validity.period", "864000", PropertyType.NUMBER, "Validity period for individual component analysis cache"),
+    SCANNER_SNYK_ENABLED("scanner", "snyk.enabled", "false", PropertyType.BOOLEAN, "Flag to enable/disable Snyk Vulnerability Analysis"),
+    SCANNER_SNYK_API_TOKEN("scanner", "snyk.api.token", null, PropertyType.ENCRYPTEDSTRING, "The API token used for Snyk API authentication"),
+    SCANNER_SNYK_ORG_ID("scanner", "snyk.org.id", null, PropertyType.STRING, "The Organization ID used for Snyk API access"),
+    SCANNER_SNYK_API_VERSION("scanner", "snyk.api.version", "2022-09-15", PropertyType.STRING, "Snyk API version"),
+    SCANNER_SNYK_CVSS_SOURCE("scanner", "snyk.cvss.source", "NVD", PropertyType.STRING, "Type of source to be prioritized for cvss calculation"),
+    SCANNER_SNYK_BASE_URL("scanner", "snyk.base.url", "https://api.snyk.io", PropertyType.URL, "Base Url pointing to the hostname and path for Snyk analysis"),
     VULNERABILITY_SOURCE_NVD_ENABLED("vuln-source", "nvd.enabled", "true", PropertyType.BOOLEAN, "Flag to enable/disable National Vulnerability Database"),
     VULNERABILITY_SOURCE_NVD_FEEDS_URL("vuln-source", "nvd.feeds.url", "https://nvd.nist.gov/feeds", PropertyType.URL, "A base URL pointing to the hostname and path of the NVD feeds"),
     VULNERABILITY_SOURCE_GITHUB_ADVISORIES_ENABLED("vuln-source", "github.advisories.enabled", "false", PropertyType.BOOLEAN, "Flag to enable/disable GitHub Advisories"),
@@ -72,17 +78,17 @@ public enum ConfigPropertyConstants {
     ACCESS_MANAGEMENT_ACL_ENABLED("access-management", "acl.enabled", "false", PropertyType.BOOLEAN, "Flag to enable/disable access control to projects in the portfolio"),
     NOTIFICATION_TEMPLATE_BASE_DIR("notification", "template.baseDir", SystemUtils.getEnvironmentVariable("DEFAULT_TEMPLATES_OVERRIDE_BASE_DIRECTORY", System.getProperty("user.home")), PropertyType.STRING, "The base directory to use when searching for notification templates"),
     NOTIFICATION_TEMPLATE_DEFAULT_OVERRIDE_ENABLED("notification", "template.default.override.enabled", SystemUtils.getEnvironmentVariable("DEFAULT_TEMPLATES_OVERRIDE_ENABLED", "false"), PropertyType.BOOLEAN, "Flag to enable/disable override of default notification templates"),
-    TASK_SCHEDULER_LDAP_SYNC_CADENCE("task-scheduler", "ldap.sync.cadence","6", PropertyType.INTEGER, "Sync cadence (in hours) for LDAP"),
-    TASK_SCHEDULER_GHSA_MIRROR_CADENCE("task-scheduler", "ghsa.mirror.cadence","24", PropertyType.INTEGER, "Mirror cadence (in hours) for Github Security Advisories"),
-    TASK_SCHEDULER_OSV_MIRROR_CADENCE("task-scheduler", "osv.mirror.cadence","24", PropertyType.INTEGER, "Mirror cadence (in hours) for OSV database"),
-    TASK_SCHEDULER_NIST_MIRROR_CADENCE("task-scheduler", "nist.mirror.cadence","24", PropertyType.INTEGER, "Mirror cadence (in hours) for NVD database"),
-    TASK_SCHEDULER_VULNDB_MIRROR_CADENCE("task-scheduler", "vulndb.mirror.cadence","24", PropertyType.INTEGER, "Mirror cadence (in hours) for VulnDB database"),
-    TASK_SCHEDULER_PORTFOLIO_METRICS_UPDATE_CADENCE("task-scheduler", "portfolio.metrics.update.cadence","1", PropertyType.INTEGER, "Update cadence (in hours) for portfolio metrics"),
-    TASK_SCHEDULER_VULNERABILITY_METRICS_UPDATE_CADENCE("task-scheduler", "vulnerability.metrics.update.cadence","1", PropertyType.INTEGER, "Update cadence (in hours) for vulnerability metrics"),
-    TASK_SCHEDULER_PORTFOLIO_VULNERABILITY_ANALYSIS_CADENCE("task-scheduler", "portfolio.vulnerability.analysis.cadence","24", PropertyType.INTEGER, "Launch cadence (in hours) for portfolio vulnerability analysis"),
-    TASK_SCHEDULER_REPOSITORY_METADATA_FETCH_CADENCE("task-scheduler", "repository.metadata.fetch.cadence","24", PropertyType.INTEGER, "Metadada fetch cadence (in hours) for package repositories"),
-    TASK_SCHEDULER_INTERNAL_COMPONENT_IDENTIFICATION_CADENCE("task-scheduler", "internal.components.identification.cadence","6", PropertyType.INTEGER, "Internal component identification cadence (in hours)"),
-    TASK_SCHEDULER_COMPONENT_ANALYSIS_CACHE_CLEAR_CADENCE("task-scheduler", "component.analysis.cache.clear.cadence","72", PropertyType.INTEGER, "Cleanup cadence (in hours) for component analysis cache");
+    TASK_SCHEDULER_LDAP_SYNC_CADENCE("task-scheduler", "ldap.sync.cadence", "6", PropertyType.INTEGER, "Sync cadence (in hours) for LDAP"),
+    TASK_SCHEDULER_GHSA_MIRROR_CADENCE("task-scheduler", "ghsa.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for Github Security Advisories"),
+    TASK_SCHEDULER_OSV_MIRROR_CADENCE("task-scheduler", "osv.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for OSV database"),
+    TASK_SCHEDULER_NIST_MIRROR_CADENCE("task-scheduler", "nist.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for NVD database"),
+    TASK_SCHEDULER_VULNDB_MIRROR_CADENCE("task-scheduler", "vulndb.mirror.cadence", "24", PropertyType.INTEGER, "Mirror cadence (in hours) for VulnDB database"),
+    TASK_SCHEDULER_PORTFOLIO_METRICS_UPDATE_CADENCE("task-scheduler", "portfolio.metrics.update.cadence", "1", PropertyType.INTEGER, "Update cadence (in hours) for portfolio metrics"),
+    TASK_SCHEDULER_VULNERABILITY_METRICS_UPDATE_CADENCE("task-scheduler", "vulnerability.metrics.update.cadence", "1", PropertyType.INTEGER, "Update cadence (in hours) for vulnerability metrics"),
+    TASK_SCHEDULER_PORTFOLIO_VULNERABILITY_ANALYSIS_CADENCE("task-scheduler", "portfolio.vulnerability.analysis.cadence", "24", PropertyType.INTEGER, "Launch cadence (in hours) for portfolio vulnerability analysis"),
+    TASK_SCHEDULER_REPOSITORY_METADATA_FETCH_CADENCE("task-scheduler", "repository.metadata.fetch.cadence", "24", PropertyType.INTEGER, "Metadada fetch cadence (in hours) for package repositories"),
+    TASK_SCHEDULER_INTERNAL_COMPONENT_IDENTIFICATION_CADENCE("task-scheduler", "internal.components.identification.cadence", "6", PropertyType.INTEGER, "Internal component identification cadence (in hours)"),
+    TASK_SCHEDULER_COMPONENT_ANALYSIS_CACHE_CLEAR_CADENCE("task-scheduler", "component.analysis.cache.clear.cadence", "72", PropertyType.INTEGER, "Cleanup cadence (in hours) for component analysis cache");
 
     private String groupName;
     private String propertyName;

diff --git a/src/main/java/org/dependencytrack/model/SnykCvssSource.java b/src/main/java/org/dependencytrack/model/SnykCvssSource.java
@@ -0,0 +1,9 @@
+package org.dependencytrack.model;
+
+public enum SnykCvssSource {
+
+    NVD,
+    SNYK,
+    RHEL,
+    SUSE
+}
diff --git a/src/main/java/org/dependencytrack/model/Vulnerability.java b/src/main/java/org/dependencytrack/model/Vulnerability.java
@@ -110,7 +110,8 @@ public enum Source {
         OSSINDEX, // Sonatype OSS Index
         RETIREJS, // Retire.js
         INTERNAL,  // Internally-managed (and manually entered) vulnerability
-        OSV // Google OSV Advisories
+        OSV, // Google OSV Advisories
+        SNYK,    // Snyk Purl Vulnerability
     }
 
     @PrimaryKey

diff --git a/src/main/java/org/dependencytrack/model/VulnerabilityAlias.java b/src/main/java/org/dependencytrack/model/VulnerabilityAlias.java
@@ -90,6 +90,13 @@ public class VulnerabilityAlias implements Serializable {
     @Pattern(regexp = RegexSequence.Definition.PRINTABLE_CHARS_PLUS, message = "The osvId field may only contain printable characters")
     private String osvId;
 
+    @Persistent
+    @Column(name = "SNYK_ID")
+    @Index(name = "VULNERABILITYALIAS_SNYK_ID_IDX")
+    @JsonDeserialize(using = TrimmedStringDeserializer.class)
+    @Pattern(regexp = RegexSequence.Definition.PRINTABLE_CHARS_PLUS, message = "The snykId field may only contain printable characters")
+    private String snykId;
+
     @Persistent
     @Column(name = "GSD_ID")
     @Index(name = "VULNERABILITYALIAS_GSD_ID_IDX")
@@ -158,6 +165,10 @@ public void setOsvId(String osvId) {
         this.osvId = osvId;
     }
 
+    public String getSnykId() { return snykId; }
+
+    public void setSnykId(String snykId) { this.snykId = snykId; }
+
     public String getGsdId() {
         return gsdId;
     }