Collect logs improvements

[TheRealFalcon]

Proposed Commit Message

feat: collect-logs improvements

* Collect sensitive data by default since we ask for it more often
  than not
* Output warning that we're collecting sensitive data
* Glob most of /run/cloud-init, /etc/cloud, and /var/lib/cloud
* Stop creating empty directories in the tarball
* Require running as root given that the logs are root read-only
* Update apport accordingly

Additional Context

Test Steps

Merge type

• Squash merge using "Proposed Commit Message"
• Rebase and merge unique commits. Requires commit messages per-commit each referencing the pull request number (#<PR_NUM>)

[holmanb]

Looks like this also fixes #5297

$ whoami
ubuntu
$ apport-cli vim

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
..

*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (5.6 KB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C)

[holmanb]

@TheRealFalcon nice work here, I've left a couple of comments inline but overall this looks good to me. Running locally I see the following files grabbed:

# tree
.
├── dmesg.txt
├── dpkg-version
├── etc
│   └── cloud
│       ├── build.info
│       ├── cloud.cfg
│       └── cloud.cfg.d
│           ├── 05_logging.cfg
│           ├── 90_dpkg.cfg
│           └── README
├── journal.txt
├── run
│   └── cloud-init
│       ├── cloud-id
│       ├── cloud-id-lxd
│       ├── cloud-init-generator.log
│       ├── cloud.cfg
│       ├── combined-cloud-config.json
│       ├── ds-identify.log
│       ├── enabled
│       ├── instance-data-sensitive.json
│       ├── instance-data.json
│       ├── result.json
│       └── status.json
├── var
│   ├── lib
│   │   └── cloud
│   │       ├── data
│   │       │   ├── instance-id
│   │       │   ├── previous-datasource
│   │       │   ├── previous-hostname
│   │       │   ├── previous-instance-id
│   │       │   ├── python-version
│   │       │   ├── result.json
│   │       │   ├── set-hostname
│   │       │   └── status.json
│   │       └── instance
│   │           ├── boot-finished
│   │           ├── cloud-config.txt
│   │           ├── datasource
│   │           ├── network-config.json
│   │           ├── obj.pkl
│   │           ├── user-data.txt
│   │           ├── user-data.txt.i
│   │           ├── vendor-data.txt
│   │           ├── vendor-data.txt.i
│   │           ├── vendor-data2.txt
│   │           └── vendor-data2.txt.i
│   └── log
│       ├── cloud-init-output.log
│       └── cloud-init.log
└── version

[holmanb]

I know that you didn't write this, but we shouldn't need this helper at all. This is the only callsite of util.chdir(), yet tar (including busybox tar) has -C which can switch directories before running the operation.

[TheRealFalcon]

Alas, it is still used in bbdeb.

[holmanb]

Alas, it is still used in bbdeb.

I'm pretty strongly opposed to shipping dead code in cloud-init that exists only for use in external utilities, but I'm also fine with saying that it's not worth dealing with right now.

[TheRealFalcon]

@holmanb , I believe I addressed all of your comments.

[holmanb]

This looks great, @TheRealFalcon. Tested, thanks for this PR!