feat(image): add progress bar for image layer pulling

[nikpivkin]

Description

TODO:

• add tests
• progress bar only works with max-image-size
• use the progress bar pool to fix flickering when pulling layers in parallel

Example:

❯ ./trivy image redis:5.0 --max-image-size 200mb
2024-12-26T23:47:36+06:00       INFO    [vuln] Vulnerability scanning is enabled
2024-12-26T23:47:36+06:00       INFO    [secret] Secret scanning is enabled
2024-12-26T23:47:36+06:00       INFO    [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-12-26T23:47:36+06:00       INFO    [secret] Please see also https://aquasecurity.github.io/trivy/dev/docs/scanner/secret#recommendation for faster secret detection
95dff52c35dd 1.35 MiB / 1.35 MiB [-------------------------------------------------------------------------------] 100.00%
c37fdfa5080c 572 B / 572 B [-------------------------------------------------------------------------------------] 100.00%
3716059b2ea5 134 B / 134 B [-------------------------------------------------------------------------------------] 100.00%
84566964bca1 1.69 KiB / 1.69 KiB [-------------------------------------------------------------------------------] 100.00%
598399fdc4dd 6.94 MiB / 6.94 MiB [-------------------------------------------------------------------------------] 100.00%
ec4a38999118 29.96 MiB / 29.96 MiB [-----------------------------------------------------------------------------] 100.00%

Related issues

• Close Show progress when fetching layers #330

Related PRs

• feat(image): prevent scanning oversized container images #8178

Remove this section if you don't have related PRs.

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[knqyf263]

This approach may help.

// progressLayer wraps a v1.Layer to add progress bar functionality
type progressLayer struct {
	v1.Layer
}

func (l *progressLayer) Compressed() (io.ReadCloser, error) {
	rc, err := l.Layer.Compressed()
	if err != nil {
		return nil, err
	}

	size, err := l.Layer.Size()
	if err != nil {
		return nil, err
	}

	bar := pb.Full.Start64(size)
	bar.Set(pb.Bytes, true)

	return bar.NewProxyReader(rc), nil
}

func NewProgressLayer(layer v1.Layer) (v1.Layer, error) {
	return partial.CompressedToLayer(&progressLayer{ Layer: layer})
}

[nikpivkin]

@knqyf263 I encountered a problem: if the --max-image-size flag is not passed, layers are processed without preloading. As a result, messages are displayed in logs during layer processing, which breaks the display of progress bars. What if we wrap the logger, and clear the progress bars before logging the message, so as not to noise the output?

[knqyf263]

Yes, so we have to show the progress bar only when --max-image-size is passed.

[nikpivkin]

OK, I will implement it only when using -max-image-size flag. By the way, I found a way to anchor the progress bars at the bottom. of the terminal and get rid of noise https://github.com/nikpivkin/pin-progress. If it is worth it, we can implement it for normal scanning in another PR.

[nikpivkin]

Adding logging in the case of loading images can also break progress bars and it can be hard to keep track of.

[knqyf263]

OK, I will implement it only when using -max-image-size flag. By the way, I found a way to anchor the progress bars at the bottom. of the terminal and get rid of noise https://github.com/nikpivkin/pin-progress. If it is worth it, we can implement it for normal scanning in another PR.

It looks like a good idea.

[nikpivkin]

@knqyf263 I'd like to discuss some details.

In order for progress to display correctly and not break when logging or creating multiple progress bars, we need to:

• Create a global progress bar container instance that manages all progress bars. For example, separate progress bars are created when extracting and scanning layers. If one of the layers initiates a java-db load with its own progress bar, they must not conflict.
• Redirect all output through the progress container so that logging does not break correct progress display.

I found a package that meets all the requirements: https://github.com/vbauerster/mpb

• Progress implements the io.Write interface and writing to the container prints above the bars. https://pkg.go.dev/github.com/vbauerster/mpb/[email protected]#Progress.Write
• Adding bars https://pkg.go.dev/github.com/vbauerster/mpb/[email protected]#Progress.AddBar

WDYT?

[knqyf263]

@nikpivkin The library looks good to me, but how hard is it to implement? We wanted to introduce progress bars to improve the UX, but if it's too hard to implement, I don't see the need to spend that much time on it.

[nikpivkin]

I actually think this is a big change for a small UX improvement.

[knqyf263]

I actually think this is a big change for a small UX improvement.

OK, then we should suspend it until we can find an easy way.