🚧 REBRANDING NOTICE 🚧

This project is currently undergoing rebranding (see https://github.com/orgs/hermetoproject/discussions/824).

You will gradually notice changes to the code base and eventually - the repository name and release artifacts. We are taking steps to ensure backwards compatibility, so feel free to continue using the project as normal, stay tuned!

Hermeto

Hermeto is a CLI tool that pre-fetches your project's dependencies to aid in making your build process hermetic.

To see if we support your package manager(s), please check the package managers section.

The primary intended use of Hermeto's outputs is for network-isolated container builds (see usage).

Table of contents

• Goals
• Installation
• Basic usage
• Configuration
• Package managers
• Project status
• Contributing

Goals

Please note that Hermeto is rather picky, aiming to:

• encourage or enforce best practices
• never execute arbitrary code - looking at you setup.py (discouraged)
• keep the implementation simple

To play nicely with Hermeto, the build process for your project must be

• Defined - Hermeto only fetches dependencies that are explicitly declared - typically in a lockfile generated by your package manager.

• Reproducible - Hermeto will refuse to fetch a dependency if it's not pinned to an exact version. This goes for transitive dependencies as well (and ties to the Defined point). Most package managers pin all dependencies automatically in lockfiles.

• Secure - Even with a lockfile, your build is not truly safe from supply chain attacks (such as dependency confusion) unless you verify the checksums of all dependencies. If your package manager supports specifying the expected checksums, we strongly encourage you to make use of them.

  ⚠ Hermeto will verify checksums if present, but doesn't require them by default. This may change in the future.

In return, Hermeto will help make your build

• Auditable - by generating a manifest of all the dependencies that go into your build.

The ability to achieve the goals depends on the hermeticity of the build process. Ideally, you should try to isolate the build from both the internet and the underlying host system to avoid implicit dependencies, irreproducible behavior and whole hosts of other issues. Hermeto itself is not a hermetic build system. We suggest you take advantage of existing technologies - such as containers - to achieve isolation (see usage).

Installation

Standalone

We do not distribute Hermeto as a standalone package as of now.

To install Hermeto for local development, see the CONTRIBUTING.md.

Container image

quay.io/konflux-ci/hermeto:latest

The container is re-built automatically on every merge to the main branch.

You may wish to set up an alias to make local usage more convenient:

alias hermeto='podman run --rm -ti -v "$PWD:$PWD:z" -w "$PWD" quay.io/konflux-ci/hermeto:latest'

Note that the alias mounts the current working directory - the container will have access to files in that directory and nowhere else.

Basic usage

hermeto fetch-deps \
  --source ./my-repo \
  --output ./hermeto-output \
  --sbom-output-type cyclonedx \
  gomod

The fetch-deps command fetches your project's dependencies and stores them on your disk. Hermeto also produces a detailed SBOM containing information about all the project's components and packages. You can find the SBOM in the output directory.

See docs/usage.md for a more detailed, practical (cough) example of Hermeto usage.

You might also like to check out hermeto --help and the --help texts of the available subcommands.

Configuration

You can change Hermeto's configuration by specifying a configuration file while invoking any of the CLI commands:

 hermeto --config-file config.yaml fetch-deps --source ./my-repo gomod

Any parameter specified in this file will override the default values present in the config.py module.

The only supported format for the config file is YAML.

Available configuration parameters

• default_environment_variables - a dictionary where the keys are names of package managers. The values are dictionaries where the keys are default environment variables to set for that package manager and the values are the environment variable values.
• gomod_download_max_tries - a maximum number of attempts for retrying go commands.
• gomod_strict_vendor - (deprecated) the bool to disable/enable the strict vendor mode. For a repo that has gomod dependencies, if the vendor directory exists and this config option is set to True, one of the vendoring flags must be used. This option no longer has any effect when set. Check the vendoring docs for more information.
• goproxy_url - sets the value of the GOPROXY variable that Hermeto uses internally when downloading Go modules. See Go environment variables.
• requests_timeout - a number (in seconds) for requests.get()'s 'timeout' parameter, which sets an upper limit on how long requests can take to make a connection and/or send a response. Larger numbers set longer timeouts.
• subprocess_timeout - a number (in seconds) to set a timeout for commands executed by the subprocess module. Set a larger number to give the subprocess execution more time.

Package managers

Package manager	Ecosystem
bundler	Ruby
cargo	Rust
generic	N/A
gomod	Go
npm	JavaScript
pip	Python
rpm*	RPM
yarn	JavaScript

*Currently, rpm is not fully supported. Therefore, we do not have documentation for it.

bundler

https://bundler.io/

Hermeto supports bundler by parsing the Gemfile.lock file present in the source repository and downloading the declared dependencies.

To generate a lockfile or to make sure the file is up to date, you can use for example the bundle lock command, which generates the Gemfile.lock file based on the dependencies specified in the Gemfile. Both files must be present in the source repository so you should check them into your git repository.

See docs/bundler.md for more details.

cargo

https://doc.rust-lang.org/cargo/

Hermeto supports Cargo by invoking the Cargo CLI to fetch Rust dependencies as declared in your project, ensuring reproducible builds with explicitly defined versions. Make sure to have up-to-date Cargo.lock present in your project.

See docs/cargo.md for more details.

generic fetcher

Generic fetcher is a way for Hermeto to support prefetching arbitrary files that don't fit into other package managers. With the generic fetcher, you can easily fetch those files with Hermeto along with your other language-specific dependencies, satisfy the hermetic build condition and have them recorded in the SBOM.

Hermeto uses a simple custom lockfile named artifacts.lock.yaml that is expected to be present in the repository, or supplied in JSON input. The lockfile describes the urls, checksums and output filenames for the downloaded files.

Currently supported types of artifacts:

• Arbitrary files
• Maven artifacts

See docs/generic.md for more details.

gomod

https://go.dev/ref/mod

Current version: 1.24 ^{1 2}

The gomod package manager works by parsing the go.mod file present in the source repository to determine which dependencies to download. Hermeto does not parse this file on its own - rather, we rely on the go command to download and list the required dependencies.

From go 1.17 onward, the go.mod file includes all the transitively required dependencies of your application - see the section about Pruned module graphs in the 1.17 changelog. In previous go versions, the go.mod file included only direct dependencies. Hermeto does support downloading and listing all transitive dependencies for earlier versions thanks to Go's backwards compatibility². Note that using go >= 1.17 in your project has the added benefit of downloading fewer dependencies (as noted in the changelog), in some cases drastically so.

See docs/gomod.md for more details.

npm

https://docs.npmjs.com/

Hermeto supports npm by parsing package-lock.json file present in the source repository and downloading the declared dependencies.

To generate lockfile or to make sure the file is up to date, you can use npm install.

Make sure lockfile version is higher than v1 (Node.js 15 or higher).

See docs/npm.md for more details.

pip

https://pip.pypa.io/en/stable/

Hermeto supports pip by parsing requirements.txt files present in the source repository and downloading the declared dependencies.

The files must be lockfiles, i.e. declare all the transitive dependencies and pin them to specific versions. Generating such a lockfile is best done using tools like pip-compile.

We support source distribution file format (sdist) as well as binary distribution file format (wheel).

See docs/pip.md for more details.

yarn

https://yarnpkg.com/

Current version: v4

Unlike NPM, hermeto merely drives the underlying yarn CLI command operations, that is, hermeto leaves most of the heavy lifting to Yarn itself and it mainly focuses on post-process validation. Note that having a Yarn lockfile (yarn.lock) checked into the repository is paramount for hermeto to process a project successfully. If missing, you can easily generate one by running yarn install prior to pointing hermeto to your project.

See docs/yarn.md for more details.

Project status

Hermeto was derived (but is not a direct fork) from Cachito.

Footnotes

1. Hermeto expects to use a specific version of the go command when downloading dependencies. This is the version installed in the hermeto container. We do not guarantee correctness if you run Hermeto locally (outside the container) with a different Go version. You are free to use a different version to build your project. ↩

2. The go command promises to be backwards compatible with previous versions. If your go.mod file specifies the intended go version, Hermeto should handle it appropriately. If your go version is higher than what Hermeto uses, there is a good chance it will be compatible regardless, as long as the dependency resolution did not change between the two versions. For example, dependency resolution did change in go 1.18 but not in go 1.19. Things are a bit more complicated with Go 1.21, if you are or have been experiencing issues with hermeto related to Go 1.21+, please refer to docs/gomod.md. ↩ ↩²