Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Docker Builder #168

Merged
merged 63 commits into from
Jan 31, 2024
Merged

Upgrade Docker Builder #168

merged 63 commits into from
Jan 31, 2024

Conversation

Byrdman32
Copy link
Member

@Byrdman32 Byrdman32 commented Jan 27, 2024

Note: This PR started as a hotfix and then branched into this builder overhaul.

Docker Images are the base of our new Autonomy Codebase, so therefore we need to have an efficient yet affective method of rebuilding them when we add new libraries or configuration preferences. And this pull request streamlines that who process.

Jammy (x64) Focal (x64) JetPack (ARM64)
Before ~ 01:00:00 hrs ~ 01:00:00 hrs ~ 03:00:00 hrs

The above table shows the average results we were seeing when having to build everything from the ground up in the event a package near the beginning of the dockerfile changed.

A few different methods were tried to execute this process:

  1. Re-organizing the dockerfile so that the packages that update most frequently are lower in the file structure. This didn't do much because you didn't always have a build cache loaded.
  2. Splitting the dockerfile into layer making it easier for docker to cache area of the image and make it quicker to build next time. The significantly sped up the build of the image; however, it made the images significantly larger due to multiple copies of the operating system.

And this leaves us with the method that is located in this pull request. By creating build instructions for the libraries that we build from source, and then converting the install into a debian package. We are able to store those debian packages in a new repository and when the container is build we just download the version we need from there and only rebuild when library updates. In addition to this by switching which version of the docker image builder package on GitHub Actions we are using, there is a new buildcache feature that makes it so that builds have a place to start from. And lastly, adding more logic into the builder scripts to make it so that it only gets run when it needs too.

Jammy (x64) Focal (x64) JetPack (ARM64)
After (Worst) ~ 00:30:00 hrs ~ 00:30:00 hrs ~ 02:00:00 hrs
After (Best) ~ 00:05:00 hrs ~ 00:05:00 hrs ~ 00:05:00 hrs
After (Average) ~ 00:07:00 hrs ~ 00:07:00 hrs ~ 00:12:00 hrs

And as shown by the results in the above table, the amount of time it takes to build docker images as been lowered significantly.

Also, the new format for running the docker builders have an added perk of a really fancy looking action workflow graph.

Old New
image image

And though upgrading the GitHub Action Packages, the node warnings (#165) have been fixed. In addition, all of our GitHub Actions have been updated to not use the deprecated method of setting step outputs.

We are also deprecating our support for Ubuntu 20.04 (Focal) due to it not being needed anymore and causing support issues.

resolves #165

@Byrdman32 Byrdman32 added bug Unexpected behaviors or issues that cause the software to malfunction or produce incorrect results. 1-Star Indicates a very easy task or challenge, suitable for beginners with minimal effort required. docker Tasks or issues related to Docker containerization, including Dockerfile configuration and environme actions Tasks or issues related to GitHub Actions or CI/CD automation workflows. labels Jan 27, 2024
@Byrdman32 Byrdman32 added this to the URC 2024 - Sprint 2b milestone Jan 27, 2024
@Byrdman32 Byrdman32 self-assigned this Jan 27, 2024
@Byrdman32 Byrdman32 requested a review from a team as a code owner January 27, 2024 21:04
@Byrdman32 Byrdman32 changed the title [HOTFIX] Add missing quotes to dokcer builder [HOTFIX] Add missing quotes to docker builder Jan 27, 2024
ClayJay3
ClayJay3 previously approved these changes Jan 28, 2024
@Byrdman32 Byrdman32 force-pushed the hotfix/docker-builder branch from 08d0f98 to 2fa44f7 Compare January 30, 2024 03:59
@Byrdman32 Byrdman32 added 5-Star Indicates a very difficult task, suitable for experts or those with extensive experience, requiring and removed 3-Star Indicates a moderate level of difficulty, requiring a good understanding of the topic or task. labels Jan 30, 2024
In 2022, GitHub Deprecated the `::set-output` method and changed the function used. It was unknowingly used in this new version of Autonomy for some actions and this commit corrects it.
@Byrdman32 Byrdman32 requested a review from ClayJay3 January 30, 2024 08:26
@Byrdman32 Byrdman32 dismissed ClayJay3’s stale review January 30, 2024 08:26

Lots of changes have been made.

@Byrdman32 Byrdman32 requested a review from ClayJay3 January 31, 2024 01:40
@Byrdman32 Byrdman32 merged commit e05aeaa into development Jan 31, 2024
22 checks passed
@Byrdman32 Byrdman32 deleted the hotfix/docker-builder branch January 31, 2024 03:31
MRDT-Software pushed a commit that referenced this pull request May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5-Star Indicates a very difficult task, suitable for experts or those with extensive experience, requiring actions Tasks or issues related to GitHub Actions or CI/CD automation workflows. bug Unexpected behaviors or issues that cause the software to malfunction or produce incorrect results. cleanup Code refactoring or cleanup tasks to remove deprecated features, improve clarity, or ensure code qua dev-environment Issues or enhancements related to setting up or maintaining the development environment. docker Tasks or issues related to Docker containerization, including Dockerfile configuration and environme enhancement Requests for new features or improvements to existing features.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade Docker Builder Action to Node.js 20
2 participants