R4.0 User Documentation Tracking

[awokd]

Qubes OS version:

R4.0

Affected TemplateVMs:

All

---

Steps to reproduce the behavior:

Go to
https://www.qubes-os.org/doc/

Expected behavior:

Be able to easily locate and differentiate R4.0 content

Actual behavior:

Not always able to locate easily

General notes:

From qubes-users discussion https://mail-archive.com/[email protected]/msg17835.html

Pending R4.0 content:

• vpn.md Update vpn.md doc #3520 @tasket
• firewall.md @adubois
• config-files.md
• software-update-vm.md - snapshots
• Windows 7 HVM Qubes Windows Tools (QWT) on R4 #3585 Windows 7 x64 HVM on R4.0 #3592 @taradiddles et al
• managing-vm-kernel.md @awokd
• man pages Create and update website man pages for CLI tools #3538 @andrewdavidwong
• document Whonix 14 template build process @awokd

See also:

• Pending Doc PRs
• [below]

---

[awokd]

On https://www.qubes-os.org/doc/software-update-vm/, I wrote "In R4.0 and higher, the template root filesystem is created in a thin pool so manual trims are no longer needed." Fact check?

In R4.0, qvm-revert-template-changes is deprecated. @marmarek said (https://mail-archive.com/[email protected]/msg04268.html) there's an implemented option to keep multiple snapshots of a template, but I can't locate it anywhere. Help?

The Updates Proxy section towards the bottom still needs work. For example, it says "(1) Services tab -> "qubes-yum-proxy" entry; check qvm-service manual for details". I'm not seeing this entry in either my R4.0 or R3.2 systems. There's an unlinked "VM secure update mechanism (forthcoming)" on the main doc page. Any ETA on that document? Might make sense to just link to it.

[awokd]

Noting for myself: https://www.qubes-os.org/doc/backup-restore/ needs Emergency 4.0 Recovery procedure developed

[marmarek]

Noting for myself: https://www.qubes-os.org/doc/backup-restore/ needs Emergency 4.0 Recovery procedure developed

It's here: https://www.qubes-os.org/doc/backup-emergency-restore-v4/

On https://www.qubes-os.org/doc/software-update-vm/, I wrote "In R4.0 and higher, the template root filesystem is created in a thin pool so manual trims are no longer needed." Fact check?

Yes.

In R4.0, qvm-revert-template-changes is deprecated. @marmarek said (https://mail-archive.com/[email protected]/msg04268.html) there's an implemented option to keep multiple snapshots of a template, but I can't locate it anywhere. Help?

qvm-volume ls fedora-26 to list available revisions, then qvm-volume revert fedora-26:root REVISION (REVISION is optional, if not given, it will use the last one)

This require revisions_to_keep to be > 0, but by default it is 0.

The Updates Proxy section towards the bottom still needs work. For example, it says "(1) Services tab -> "qubes-yum-proxy" entry; check qvm-service manual for details". I'm not seeing this entry in either my R4.0 or R3.2 systems. There's an unlinked "VM secure update mechanism (forthcoming)" on the main doc page. Any ETA on that document? Might make sense to just link to it.

There are two services (qvm-service, service framework):

1. qubes-updates-proxy (and its deprecated name: qubes-yum-proxy) - a service providing a proxy for templates - by default enabled in NetVMs (especially: sys-net)
2. updates-proxy-setup (and its deprecated name: yum-proxy-setup) - use a proxy provided by other VM (instead of downloading updates directly), enabled by default in all templates

This is generally the same in R3.2 and R4.0 - in both cases both old and new names works. And in both cases defaults listed above are applied if service is not explicitly listed in services tab.

The main difference between R3.2 and R4.0 here is how templates are connected to updates proxy:

• R3.2: it's network connection, first proxy on the network path is used, according to template's netvm setting
• R4.0: it's qrexec connection, proxy is configured in qrexec policy: /etc/qubes-rpc/policy/qubes.UpdatesProxy (by default configured to sys-net and/or sys-whonix, depending on firstboot choices); templates are not connected to any netvm

Example policy file in R4.0 (when whonix installed, but not set as default updatevm for all templates):

# any VM with tag `whonix-updatevm` should use `sys-whonix`; this tag is added to `whonix-gw` and `whonix-ws` during installation and is preserved during template clone
$tag:whonix-updatevm $default allow,target=sys-whonix
$tag:whonix-updatevm $anyvm deny

# other templates use sys-net
$type:TemplateVM $default allow,target=sys-net
$anyvm $anyvm deny

As for "VM secure update mechanism (forthcoming)", originally there was a plan for dom0-like update mechanism for templates too, but it was abandoned and the above is done instead.

[awokd]

"This require revisions_to_keep to be > 0, but by default it is 0." Is this a WIP? I found #3256 and tried a few commands like:
qvm-pool -i lvm -o revisions_to_keep=2
and
qvm-pool -a lvmsnap lvm_thin -o volume_group=qubes_dom0,thin_pool=pool00,revisions_to_keep=2
qvm-clone -P lvmsnap debian-9 debian-9snap
but in both cases
qvm-volume ls debian-9
qvm-volume ls debian-9snap
report REVERT_POSSIBLE=NO and looking at qubes.xml directly shows revisions_to_keep=0 everywhere (except the varlibqubes file driver pool which says revisions_to_keep=1).

Thank you and @andrewdavidwong for the other info, am updating the docs accordingly.

[marmarek]

Yes, it's broken right now...

[awokd]

The Templates: Fedora* and Templates: Debian look all set for R4.0 thanks to other contributors.
Templates: Ubuntu has no version specific content. I'm leaving updates for the remaining Templates (Archlinux, Whonix, Pentesting*) up to their respective subject matter experts.

[awokd]

Is Windows 7 on Qubes R4.0 going to be a supported configuration?

[awokd]

TODO: Pending finalization of #3260

vpn.md doc #3520 @tasket
firewall.md doc QubesOS/qubes-doc#552 @awokd
config-files.md doc https://www.qubes-os.org/doc/config-files/ (update descriptions as needed)
http-filtering-proxy.md doc https://www.qubes-os.org/doc/config/http-filtering-proxy/ (review and update as needed)

[awokd]

TODO: Split tool man pages into subdirectories (https://mail-archive.com/[email protected]/msg18387.html). I think you have this one @andrewdavidwong? Please let me know if I should open a separate issue on it; I can't seem to figure out how to add those nifty checkboxes to an existing issue like this one.

[andrewdavidwong]

TODO: Split tool man pages into subdirectories (https://mail-archive.com/[email protected]/msg18387.html). I think you have this one @andrewdavidwong?

Yes, working on it now.

Please let me know if I should open a separate issue on it;

Not necessary; this issue will do.

I can't seem to figure out how to add those nifty checkboxes to an existing issue like this one.

- [x] Foo
- [ ] Bar

renders as:

• Foo
• Bar

[tlaurion]

I didn't fell upon documentation explaining how to create snapshots of templates/qube. Is that missing or I missed it?

@marmarek: Is there an open issue for this comment?

Yes, it's broken right now...

[tlaurion]

@andrewdavidwong: This issue applies also to 4.0.

[andrewdavidwong]

@andrewdavidwong: This issue applies also to 4.0.

Which issue are you referring to? The one we're commenting in (#3495)? If so, please be aware that we use the "Documentation/website" milestone for all issues that primarily concern the documentation or website, even if they also pertain to a specific Qubes OS version, since each GitHub issue can have only one milestone.

[awokd]

@tlaurion For R3.2 snapshots, see qvm-template-commit in https://www.qubes-os.org/doc/template-implementation and "how to revert" in https://www.qubes-os.org/doc/software-update-vm/. For R4.0, because functionality is still being designed (#3256), there's no documentation yet. We're tracking it on here so it doesn't get missed once it's added.

[awokd]

firewall.md @adubois is providing additional 4.0 updates and detail

[awokd]

Windows 7 - no current maintainer (https://www.mail-archive.com/[email protected]/msg02808.html). Will leave related documentation updates for Windows 7 on R4.0 to future maintainer.

[m4lu]

I would like to suggest versioning the documentation for each release. This could lead to URLs such as:

• https://qubes-os.org/doc/R4.0/copy-from-dom0/
• https://qubes-os.org/doc/R3.2/vpn

This would make it easier for the users to judge if the currently looked at link is the one for their version of Qubes. Furthermore the currently used structure of the documentation tree can be kept with the benefit of easy 'imports' into the directory of the the next mayor release. While at that it can be made sure that all imported sections still apply for the next version.

I acknowledge that it would make the situation messier in terms of links on the website.

[andrewdavidwong]

@m4lu:

I would like to suggest versioning the documentation for each release. [...]

This has already been discussed quite a bit:
https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion

[awokd]

Completed review of User Documentation. Apart from the Pending items listed up top, I'm leaving any R4.0 updates (if necessary) of the following up to their subject matter experts:

• Templates (Archlinux, Ubuntu, Whonix)
• Pentesting*
• NetBSD
• ZFS
• Customization Guides (Fedora, Windows 7, KDE, i3, Language Localization)
• Reference Pages

The Troubleshooting section will need to be added to as problems and solutions are developed.

[awokd]

@andrewdavidwong I'll skim through the Developer Documentation section and hit what I can, but most of those are going to require a developer to address. Should I/you re-title this issue to "R4.0 User Documentation Tracking" in case you want to track that separately?
On a side note, I noticed the Security Information sections in User and Developer is mostly duplicated.

[andrewdavidwong]

I'll skim through the Developer Documentation section and hit what I can, but most of those are going to require a developer to address. Should I/you re-title this issue to "R4.0 User Documentation Tracking" in case you want to track that separately?

Sure, that's fine.

On a side note, I noticed the Security Information sections in User and Developer is mostly duplicated.

That's intentional. It's only the links, not the pages or content, that are duplicated. They appear under both because they apply to both users and developers.

[ghost]

[I didn't know where to post this - whether here on in #3592 ; we can of course move the discussion there]

Re- windows hvm installation doc: judging by the positive comments/ML posts it seems the instructions won't change much from now on. I will only have sparse free time in the next 2-3 weeks so it'd be nice to settle on how (and where) you guys think the documentation should be pushed to the official docs (see the issue's "To be discussed" section). That way I'll have time to finalize the doc, change the markdown format to Qubes coding style and submit a PR before the final 4.0 release.

IMHO:

• there should be a dedicated "clean Windows HVM" page with instructions for both R3.2 and R4.x
• the instructions are for Win7 x64 VMs (because that's the version supported by the qubes tools) but they should probably work with other Windows variants; unfortunately I really don't have time to try installing other versions. (-> add a note at the top of the page that the instructions may work with other versions, and/or post to the ML to ask for users' feedback ?).
• instructions specific to windows VMs in the hvm doc should be removed.

[awokd]

@andrewdavidwong 's call, but those all sound like good ideas to me. I remember having to bounce between https://www.qubes-os.org/doc/hvm/, https://www.qubes-os.org/doc/windows-appvms/, and https://www.qubes-os.org/doc/windows-tools-3/ when setting up a Win7 VM for the first time and it was hard to follow.

[andrewdavidwong]

there should be a dedicated "clean Windows HVM" page with instructions for both R3.2 and R4.x

I don't understand the "clean" part. ("Clean" as opposed to what?)

Other than that, sounds fine.

the instructions are for Win7 x64 VMs (because that's the version supported by the qubes tools) but they should probably work with other Windows variants; unfortunately I really don't have time to try installing other versions. (-> add a note at the top of the page that the instructions may work with other versions, and/or post to the ML to ask for users' feedback ?).

Yes, it would be fine to say that the instructions may work on other versions but haven't been tested, and that users should only attempt them on other versions at their own risk.

instructions specific to windows VMs in the hvm doc should be removed.

Agreed.

[ghost]

I don't understand the "clean" part. ("Clean" as opposed to what?)

As opposed to importing from R3.2. Probably not the right word - false friend with French and Bulgarian languages. Removed...

Yesterday I "forked" the issue's instructions to include R3.2 instructions (since the issue is only for R4.0) ; fyi new text here. If you agree I'll remove the "For the impatient" section (tl;dr; in the issue) since it is a bit of a mess on R3.2.

I don't have a R3.2 install to see a newly created VM's xml file after the following instructions ; can you or @awokd send one ? (I need it to write the appropriate sed command that switches the video adapter from std vga to cirrus).

qvm-create win7new --hvm --label red
qvm-prefs win7new memory 4000
qvm-prefs win7new maxmem 4000
qvm-prefs win7new kernel ''
qvm-volume extend win7new:root 25g
qvm-prefs win7new debug true

[awokd]

@taradiddles uploaded to your wiki and changed some of the commands to their R3.2 equivalent.

[ghost]

@awokd - thank you, I'll update the instructions in the "detailed" section.
btw sorry for the discrepancies with R3.2 commands, I planned to double-check the syntax yesterday and then totally forgot.

[andrewdavidwong]

If you agree I'll remove the "For the impatient" section (tl;dr; in the issue) since it is a bit of a mess on R3.2.

I'm afraid I'm not in a position to judge that. I suggest basing your decision on what you think will be most useful for the users who are likely to read the page.

[adubois]

@awokd, I've created PR 605 to address firewall.md doc task. Not sure how to tick it at the top...

[awokd]

Thanks for writing that up @adubois! Added the link. I'm keeping an eye on the PRs for qubes-doc; I have a couple in there pending too. Once they get merged I'll check the boxes off.

[andrewdavidwong]

#3538 is done.

[awokd]

From what I can tell, the existing Salt documentation is compatible with 4.0 so I'm removing it from this list. It could still use some improvement, though.

[tabbyrobin]

Not sure if right thread for this, but is “Copy Dom0 clipboard” feature deprecated in R4? This page says it's available for 3.2, but does not mention 4.0: https://github.com/QubesOS/qubes-doc/blob/master/common-tasks/copy-from-dom0.md

[awokd]

Not sure if right thread for this, but is “Copy Dom0 clipboard” feature deprecated in R4?

Yes, deprecated. The "other versions" procedure in #3 works for 4.0, though.

[awokd]

Going to close this tracking ticket as well per QubesOS/qubes-doc#637, since that was the last thing on here.