Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automate the main bulk of our release tasks #1016

Closed
apyrgio opened this issue Nov 28, 2024 · 0 comments · Fixed by #1023
Closed

Automate the main bulk of our release tasks #1016

apyrgio opened this issue Nov 28, 2024 · 0 comments · Fixed by #1023
Assignees
@apyrgio
Labels
enhancement New feature or request
Milestone
0.9.0

Comments

@apyrgio
Copy link
Contributor

apyrgio commented Nov 28, 2024 •
edited
Loading

Our release tasks, as described in our RELEASE.md, take a lot of time to run, mostly for three reasons:

  1. Waiting for a command to finish. Some of our commands, such as building images or development environments, take a few minutes to finish. Usually by then, the developer may switch to another task, and check the running command irregularly. The context switching cost and the drift between the command's end adds a lot of time to the release.
  2. Commands run sequentially. There are some things that we need to do in order to prepare artifacts for the release (such as building the container image) and other things that can be done in parallel (such as building the Fedora RPMs for each version). Currently, these happen sequentially because the destination dir of the artifacts is the hardcoded dist/ folder in our repo, and parallel runs may overwrite the contents in it.
  3. Environment state. Sometimes our build environment may be in a state where it's not clean, or there's no space. In these cases, we have to debug errors with our artifacts or our commands, which cost us some time.

Automating the main bulk of our release tasks, letting them run in parallel, and having a way to bring our build environment to a clean state will be huge wins, until we have a way to build the artifacts in isolated runners.
@apyrgio apyrgio added the enhancement New feature or request label Nov 28, 2024
@apyrgio apyrgio added this to the 0.9.0 milestone Nov 28, 2024
@almet almet moved this from Todo to In Progress in Dangerzone ✨ Nov 28, 2024
@almet almet moved this from In Progress to PR Review in Dangerzone ✨ Dec 3, 2024
@almet almet moved this from PR Review to In Progress in Dangerzone ✨ Dec 3, 2024
apyrgio added a commit that referenced this issue Dec 4, 2024
@apyrgio
Automate a large portion of our release tasks
52eae7c 
Create a `dodo.py` file where we define the dependencies and targets of
each release task, as well as how to run it. Currently, we have
automated all of our Linux and macOS tasks, except for adding Linux
packages to the respective APT/YUM repos.

The tasks we have automated follow below:

    build_image               Build the container image using ./install/common/build-image.py
    check_container_runtime   Test that the container runtime is ready.
    clean_container_runtime   Clean the storage space of the container runtime.
    clean_prompt              Make sure that the user really wants to run the clean tasks.
    debian_deb                Build a Debian package for Debian Bookworm.
    debian_env                Build a Debian Bookworm dev environment.
    download_tessdata         Download the Tesseract data using ./install/common/download-tessdata.py
    fedora_env                Build Fedora dev environments.
    fedora_env:40             Build Fedora 40 dev environments
    fedora_env:41             Build Fedora 41 dev environments
    fedora_rpm                Build Fedora packages for every supported version.
    fedora_rpm:40             Build a Fedora 40 package
    fedora_rpm:40-qubes       Build a Fedora 40 package for Qubes
    fedora_rpm:41             Build a Fedora 41 package
    fedora_rpm:41-qubes       Build a Fedora 41 package for Qubes
    git_archive               Build a Git archive of the repo.
    init_release_dir          Create a directory for release artifacts.
    macos_build_dmg           Build the macOS .dmg file for Dangerzone.
    macos_check_cert          Test that the Apple developer certificate can be used.
    macos_check_system        Run macOS specific system checks, as well as the generic ones.
    poetry_install            Setup the Poetry environment

Closes #1016
@apyrgio apyrgio mentioned this issue Dec 4, 2024
Merged
@apyrgio apyrgio moved this from In Progress to PR Review in Dangerzone ✨ Dec 5, 2024
@apyrgio apyrgio removed the status in Dangerzone ✨ Dec 5, 2024
@apyrgio apyrgio moved this to PR Review in Dangerzone ✨ Dec 10, 2024
apyrgio added a commit that referenced this issue Dec 10, 2024
@apyrgio
Automate a large portion of our release tasks
b6a5556 
Create a `dodo.py` file where we define the dependencies and targets of
each release task, as well as how to run it. Currently, we have
automated all of our Linux and macOS tasks, except for adding Linux
packages to the respective APT/YUM repos.

The tasks we have automated follow below:

    build_image               Build the container image using ./install/common/build-image.py
    check_container_runtime   Test that the container runtime is ready.
    clean_container_runtime   Clean the storage space of the container runtime.
    clean_prompt              Make sure that the user really wants to run the clean tasks.
    debian_deb                Build a Debian package for Debian Bookworm.
    debian_env                Build a Debian Bookworm dev environment.
    download_tessdata         Download the Tesseract data using ./install/common/download-tessdata.py
    fedora_env                Build Fedora dev environments.
    fedora_env:40             Build Fedora 40 dev environments
    fedora_env:41             Build Fedora 41 dev environments
    fedora_rpm                Build Fedora packages for every supported version.
    fedora_rpm:40             Build a Fedora 40 package
    fedora_rpm:40-qubes       Build a Fedora 40 package for Qubes
    fedora_rpm:41             Build a Fedora 41 package
    fedora_rpm:41-qubes       Build a Fedora 41 package for Qubes
    git_archive               Build a Git archive of the repo.
    init_release_dir          Create a directory for release artifacts.
    macos_build_dmg           Build the macOS .dmg file for Dangerzone.
    macos_check_cert          Test that the Apple developer certificate can be used.
    macos_check_system        Run macOS specific system checks, as well as the generic ones.
    poetry_install            Setup the Poetry environment

Closes #1016
@github-project-automation github-project-automation bot moved this from PR Review to Done in Dangerzone ✨ Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@apyrgio apyrgio

Labels
enhancement New feature or request
Projects
Dangerzone ✨
Status: Done
Milestone
0.9.0
Development

Successfully merging a pull request may close this issue.

Automate a large portion of our release tasks freedomofpress/dangerzone
1 participant
@apyrgio