diff --git a/README.md b/README.md index 8af429b6..e3b7537f 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ action on sources, like cloning and verifying Git repos, rendering a SPEC file, generating SRPM or Debian source packages, a new cage is used. Only the signing, publishing, and uploading processes are executed locally outside a cage. (This will be improved in the future.) For now, only Docker, Podman, -Local, and Qubes executors are available. +Local, Qubes and Windows executors are available. ## Dependencies @@ -131,6 +131,63 @@ $ qvm-prefs work-qubesos default_dispvm qubes-builder-dvm ``` +## Windows executors and building Windows Tools + +There are two different Windows executors: `SSHWindowsExecutor` and `WindowsQubesExecutor`. +Prerequisites for both executors are a superset of the Qubes executor (see above). +For code signing you need `osslsigncode` installed in the Linux disposable template +and the signing vault qube (see below). + +### `SSHWindowsExecutor` + +This executor is meant for development, it uses SSH to communicate with a Windows system +that is used for building. Scripts that can automatically create such qube are found +in the `tools/windows` directory (they require `genisoimage` installed in the disposable template +(`qubes-builder-dvm`)). You will need an unmodified Windows 10/11 installation iso and about +50GiB of disk space. + +First, create an edited installation iso by running `generate-iso.sh`. This script +downloads prerequisites (OpenSSH server for Windows and Microsoft EWDK iso) and prepares +the installation image for unattended Windows installation. After the image is generated, +run `dom0/create-vm.sh` in dom0 to actually create and configure the worker qube (passing +the generated installation image as a `--iso` parameter). After the script finishes, the worker +qube is ready to use by the builder. + +The worker qube has outbound network connections blocked in the firewall, this is configured +by the `create-vm.sh` script. An ssh key is generated by the `generate-iso.sh` script, +the private part is saved in `~/.ssh/win-build.key` by default while the public part +is copied to the generated Windows installation image. + +If the `ssh-vm` option is set in the `builder.yml` (see below), the SSH executor automatically +starts the given vm. This option also requires the `ewdk` option, that specifies path to EWDK iso +(in `work-qubesos`). Is `ssh-vm` it's assumed that the SSH machine is configured manually. + +You can also use any SSH-accessible Windows machine instead. + +### `WindowsQubesExecutor` + +This executor works in the same manner as the Linux Qubes executor. It requires a Windows +disposable template with Qubes Windows Tools installed (you can use the SSH executor first +to build QWT). + +### General information + +It is recommended to turn off Microsoft Defender in the worker qube (especially real-time +protection) because it slows down building significantly. This is not done during unattended +setup because there is no supported way for automating this. (TODO: it enables itself after +restart which is a pain for dispvms). + +A separate vault-type qube is needed for code-signing Windows binaries. Let's assume it's named +`vault-sign`. This qube has access to actual signing keys used, either production ones (TODO: +in a HSM), or ephemeral self-signed keys. Communication with the `vault-sign` qube goes through +Qubes RPC: install RPC service scripts from `rpc/qubes.WinSign.*` in the vault qube (make sure +they are permanent in `/etc/qubes-rpc`, see [bind dirs](https://www.qubes-os.org/doc/bind-dirs/)). +`qubesbuilder.WinSign.Timestamp` needs to be installed in the default Linux disposable template +(`qubes-builder-dvm`). You also need to configure RPC policy in `dom0`, copy +`rpc/policy/51-qubesbuilder-windows.policy` to `/etc/qubes/policy.d` there (make sure the qube +names are correct). + + ## Build stages The build process consists of the following stages: @@ -160,9 +217,11 @@ Currently, only these are used: - `source` --- Manages general distribution sources - `source_rpm` --- Manages RPM distribution sources - `source_deb` --- Manages Debian distribution sources +- `source_windows` --- Manages Windows sources - `build` --- Manages general distribution building - `build_rpm` --- Manages RPM distribution building - `build_deb` --- Manages Debian distribution building +- `build_windows` --- Manages Windows building (Visual Studio solutions) - `sign` --- Manages general distribution signing - `sign_rpm` --- Manages RPM distribution signing - `sign_deb` --- Manages Debian distribution signing @@ -390,9 +449,10 @@ We provide the following list of available keys: - `vm` --- `vm` package set content. - `rpm` --- RPM plugins content. - `deb` --- Debian plugins content. -- `source` --- Fetch and source plugins (`fetch`, `source`, `source_rpm`, and - `source_deb`) content. -- `build` --- Build plugins content (`build`, `build_rpm`, and `build_deb`). +- `windows` --- Windows plugin content. +- `source` --- Fetch and source plugins (`fetch`, `source`, `source_rpm`, + `source_deb` and `source_windows`) content. +- `build` --- Build plugins content (`build`, `build_rpm`, `build_deb` and `build_windows`). - `create-archive` --- Create source component directory archive (default: `True` unless `files` is provided and not empty). - `commands` --- Execute commands before plugin or distribution tools @@ -429,6 +489,17 @@ Here is a non-exhaustive list of distribution-specific keys: - `host-fc32` --- Fedora 32 for the `host` package set content only - `vm-bullseye` --- Bullseye for the `vm` package set only +`build_windows` specific: all output artifacts for a component need to be specified in +`.qubesbuilder` as lists of paths (relative to component root) with the following keys: +- `bin` --- binaries (`.exe`, `.dll`, `.sys` and all files that can be PE-signed) +- `inc` --- devel header files that are dependencies for other components +- `lib` --- linker libraries that are dependencies for other components + +`skip-test-sign` option can be specified to provide a list of binaries that should not be +signed with a test key (only used for the final installer binary currently, since +the self-signed certificate is in the installer itself so the binary can't be verified +if test-signed). + Inside each top level, it defines what plugin entry points like `rpm`, `deb`, and `source` will take as input. Having both `PACKAGE_SET` and `PACKAGE_SET-DISTRIBUTION_NAME` with common keys means that it is up to the @@ -449,6 +520,8 @@ currently-supported placeholders: - `@DISTFILES_DIR@` --- Replaced by `/builder/distfiles` (inside a cage) - `@SOURCE_DIR@` --- Replaced by `/builder/` (inside a cage where, `` is the component directory name) +- `@CONFIGURATION@` --- `build_windows` specific, replaced by the project configuration + (`Debug` / `Release`) ### Examples @@ -675,6 +748,32 @@ distribution tools like `dpkg-*`. This is only available for Debian distributions and not RPM distributions, as similar processing is currently not needed. +Here is an example for a Windows component (`core-vchan-xen`): + +```yaml +host: + rpm: + build: + - rpm_spec/libvchan.spec +vm: + rpm: + build: + - rpm_spec/libvchan.spec +(...) + windows: + build: + - windows/vs2022/core-vchan-xen.sln + bin: + - windows/vs2022/x64/@CONFIGURATION@/libvchan/libvchan.dll + inc: + - windows/include/libvchan.h + lib: + - windows/vs2022/x64/@CONFIGURATION@/libvchan/libvchan.lib +``` + +The `build` stage specifies a Visual Studio solution to be built. `bin`, `inc` and `lib` keys +specify output artifacts. + ## Qubes builder configuration Options available in `builder.yml`: @@ -742,7 +841,7 @@ Options available in `builder.yml`: - `templates: str` --- Testing repository for templates at publish stage. This is either `templates-itl-testing` or `templates-community-testing`. - `executor: Dict` --- Specify default executor to use. - - `type: str` --- Executor type: qubes, docker, podman or local. + - `type: str` --- Executor type: qubes, docker, podman, local or windows. - `options: Dict`: - `image: str` --- Container image to use. Specific to docker or podman type. - `dispvm: str` --- Disposable template VM to use (use `"@dispvm"` to use the calling qube `default_dispvm` property or specify a name). @@ -750,6 +849,19 @@ Options available in `builder.yml`: - `clean: bool` --- Clean container, disposable qube or temporary local folder (default `true`). - `clean-on-error: bool` --- Clean container, disposable qube or temporary local folder if any error occurred. Default is value set by `clean`. +- Options specific to the `windows` and `windows-ssh` executors (see `example-configs/windows-tools.yml`): + - `user: str` --- Name of the user account in the worker Windows machine/VM (default: `user`). + - `threads: int` --- Number of parallel threads to use for MSBuild (default: 1). + - `ewdk: str` --- Path to the EWDK iso file that will be attached to the worker qube. + +- Options specific to the `windows` executor: + - `dispvm: str` --- Name of the disposable Windows template (default: `win-build`). + +- Options specific to the `windows-ssh` executor: + - `ssh-key-path: str` --- Path to the private ssh key used for communication with the worker machine (default: `~/.ssh/win-build.key`). + - `ssh-ip: str` --- IP address to use when connecting to the worker machine. + - `ssh-vm: str` --- Name of the worker qube (optional). If specified, this qube is started automatically and the EWDK iso is attached to it as a block device. + - `stages: List[str, Dict]` --- List of stages to trigger. - `: str` --- Stage name. - `: Dict` --- Stage name provided as dict to override executor to use. @@ -859,3 +971,33 @@ For the `fetch` stage, the Qubes executor with disposable template `qubes-builde For the `build` stage of `vm-fc42`, the Podman executor with container image `fedoraimg` will be used. For the `sign` stage, the Qubes executor with disposable template `signing-access-dvm` will be used for both `vm-fc42` and `vm-jammy` For the `prep` stage of `vm-jammy`, the Local executor with base directory `/some/path` will be used. + +### Windows-specific build stage options + +Options related to Qubes Windows Tools can be specified under the `build` stage of a Windows distribution, like this: + +```yaml +distributions: + - vm-win10: + stages: + - build: + configuration: release + sign-qube: vault-sign + sign-key-name: "Qubes Windows Tools" + test-sign: true + executor: + type: windows # or windows-ssh + options: + user: user + threads: 1 + dispvm: win-build + ewdk: "dev:loop1" + #ssh-key-path: /home/user/.ssh/win-build.key + #ssh-ip: 10.137.0.20 +``` + + - `configuration: str` --- build configuration (`debug` / `release`) (default: `release`). + - `sign-qube: str` --- name of the vault qube performing code signing, see the Windows executor description above. + - `sign-key-name: str` --- name of the signing key to use. For test keys this becomes the subject of the self-signed certificate. + - `test-sign: bool` --- code signing type, `true` (default) or `false`. Test signing generates ephemeral self-signed + keys for each component. Production signing uses an already existing key signed by a public CA (TODO: HSM). diff --git a/example-configs/windows-tools.yml b/example-configs/windows-tools.yml new file mode 100644 index 00000000..bfe1ab75 --- /dev/null +++ b/example-configs/windows-tools.yml @@ -0,0 +1,59 @@ +git: + prefix: omeg/qubes- + branch: omeg/builder-v2 + maintainers: + # omeg + - 'CE8060B48282B234AE0A7815D32BF219E67BA830' + +#increment-devel-versions: true + +debug: true +verbose: true + +skip-git-fetch: false + +# this is for anything other than building, so source fetching etc +executor: + type: qubes + options: + dispvm: qubes-builder-dvm + +# dev only +less-secure-signed-commits-sufficient: + - vmm-xen-windows-pvdrivers + - core-vchan-xen + - windows-utils + - core-qubesdb + - core-agent-windows + - gui-common + - gui-agent-windows + - installer-windows-tools + +distributions: + - vm-win10: + stages: + - build: + configuration: release + sign-qube: vault-sign + sign-key-name: "Qubes Windows Tools" + test-sign: true + executor: + #type: windows-ssh + type: windows + options: + dispvm: win-build + user: user + ewdk: tools/windows/ewdk.iso + threads: 1 + #ssh-ip: 10.137.0.20 + #ssh-key-path: /home/user/.ssh/win-build.key + +components: + - vmm-xen-windows-pvdrivers + - core-vchan-xen + - windows-utils + - core-qubesdb + - core-agent-windows + - gui-common + - gui-agent-windows + - installer-windows-tools diff --git a/qubesbuilder/common.py b/qubesbuilder/common.py index 99959e3c..f711c3d6 100644 --- a/qubesbuilder/common.py +++ b/qubesbuilder/common.py @@ -105,7 +105,10 @@ def sanitize_line(untrusted_line: bytes): if 0x20 <= c <= 0x7E: pass else: - line[i] = 0x2E + if c == 0x0D: # windows newline + line[i] = 0x20 + else: + line[i] = 0x2E return bytearray(line).decode("ascii") diff --git a/qubesbuilder/config.py b/qubesbuilder/config.py index a40d82eb..ba13b1ff 100644 --- a/qubesbuilder/config.py +++ b/qubesbuilder/config.py @@ -30,7 +30,11 @@ from qubesbuilder.executors import ExecutorError from qubesbuilder.executors.container import ContainerExecutor from qubesbuilder.executors.local import LocalExecutor -from qubesbuilder.executors.qubes import LinuxQubesExecutor +from qubesbuilder.executors.qubes import ( + LinuxQubesExecutor, + WindowsQubesExecutor, +) +from qubesbuilder.executors.windows import SSHWindowsExecutor from qubesbuilder.pluginmanager import PluginManager from qubesbuilder.plugins import ( DistributionPlugin, @@ -572,6 +576,10 @@ def get_executor(options): executor = LocalExecutor(**executor_options) # type: ignore elif executor_type == "qubes": executor = LinuxQubesExecutor(**executor_options) # type: ignore + elif executor_type == "windows": + executor = WindowsQubesExecutor(**executor_options) # type: ignore + elif executor_type == "windows-ssh": + executor = SSHWindowsExecutor(**executor_options) # type: ignore else: raise ExecutorError("Cannot determine which executor to use.") return executor diff --git a/qubesbuilder/distribution.py b/qubesbuilder/distribution.py index 423d6d5a..e2ab4df9 100644 --- a/qubesbuilder/distribution.py +++ b/qubesbuilder/distribution.py @@ -42,6 +42,11 @@ DEBIAN_ARCHITECTURE = {"x86_64": "amd64", "ppc64le": "ppc64el"} +WINDOWS = { + "win10": ("windows", "10"), + "win11": ("windows", "11"), +} + class QubesDistribution: def __init__(self, distribution: str, **kwargs): @@ -65,6 +70,7 @@ def __init__(self, distribution: str, **kwargs): is_ubuntu = UBUNTU.get(self.name, None) is_archlinux = self.name == "archlinux" is_gentoo = self.name == "gentoo" + is_windows = WINDOWS.get(self.name, None) if is_fedora: self.fullname = "fedora" self.version = is_fedora.group(1) @@ -101,6 +107,10 @@ def __init__(self, distribution: str, **kwargs): self.version = "rolling" self.tag = "gentoo" self.type = "gentoo" + elif is_windows: + self.fullname, self.version = WINDOWS[self.name] + self.tag = "windows" + self.type = "windows" else: raise DistributionError( f"Unsupported distribution '{self.distribution}'." @@ -140,3 +150,8 @@ def is_archlinux(self) -> bool: def is_gentoo(self) -> bool: return self.name == "gentoo" + + def is_windows(self) -> bool: + if WINDOWS.get(self.name, None): + return True + return False diff --git a/qubesbuilder/executors/qrexec.py b/qubesbuilder/executors/qrexec.py new file mode 100755 index 00000000..8ed0c02f --- /dev/null +++ b/qubesbuilder/executors/qrexec.py @@ -0,0 +1,154 @@ +# The Qubes OS Project, http://www.qubes-os.org +# +# Copyright (C) 2021 Frédéric Pierret (fepitre) +# Copyright (C) 2025 Rafał Wojdyła +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# SPDX-License-Identifier: GPL-3.0-or-later +import re +import subprocess +from typing import List, Optional + +from qubesbuilder.common import sanitize_line +from qubesbuilder.executors import ExecutorError + + +def qrexec_call( + log, + what: str, + vm: str, + service: str, + args: Optional[List[str]] = None, + capture_output: bool = False, + options: Optional[List[str]] = None, + stdin: bytes = b"", + ignore_errors: bool = False, +) -> Optional[bytes]: + cmd = [ + "/usr/lib/qubes/qrexec-client-vm", + ] + + if options: + cmd += options + + cmd += [ + "--", + vm, + service, + ] + + if args: + cmd += args + + admin = service.startswith("admin.") + capture_output = capture_output or admin + try: + log.debug(f"qrexec call ({what}): {' '.join(cmd)}") + proc = subprocess.run( + cmd, + check=not ignore_errors, + capture_output=capture_output, + input=stdin, + ) + except subprocess.CalledProcessError as e: + if e.stderr is not None: + content = sanitize_line(e.stderr.rstrip(b"\n")).rstrip() + else: + content = str(e) + msg = f"Failed to {what}: {content}" + raise ExecutorError(msg, name=vm) + + if capture_output: + stdout = proc.stdout + if admin: + if not stdout.startswith(b"0\x00"): + stdout = stdout[2:].replace(b"\x00", b"\n") + + if not ignore_errors: + raise ExecutorError( + f"Failed to {what}: qrexec call failed: {stdout.decode("ascii", "strict")}" + ) + else: + log.debug( + f"Failed to {what}: qrexec call failed: {stdout.decode("ascii", "strict")}" + ) + stdout = stdout[2:] + return stdout + return None + + +def create_dispvm(log, template: str) -> str: + stdout = qrexec_call( + log=log, + what="create disposable qube", + vm=template, + service="admin.vm.CreateDisposable", + ) + + assert stdout + if not re.match(rb"\Adisp(0|[1-9][0-9]{0,8})\Z", stdout): + raise ExecutorError("Failed to create disposable qube.") + try: + return stdout.decode("ascii", "strict") + except UnicodeDecodeError as e: + raise ExecutorError(f"Failed to obtain disposable qube name: {str(e)}") + + +def start_vm(log, vm: str): + qrexec_call( + log=log, + what="start vm", + vm=vm, + service="admin.vm.Start", + ) + + +def vm_state(log, vm: str) -> str: + response = qrexec_call( + log=log, + what="query vm state", + vm=vm, + service="admin.vm.CurrentState", + ) + + assert response is not None + for state in response.decode("ascii", "strict").split(): + assert "=" in state + kv = state.split("=") + if kv[0] == "power_state": + return kv[1] + + raise ExecutorError( + f"Invalid response from admin.vm.CurrentState for '{vm}'" + ) + + +def kill_vm(log, vm: str): + qrexec_call( + log=log, + what="kill vm", + vm=vm, + service="admin.vm.Kill", + ignore_errors=True, + ) + + +def remove_vm(log, vm: str): + qrexec_call( + log=log, + what="remove vm", + vm=vm, + service="admin.vm.Remove", + ) diff --git a/qubesbuilder/executors/qubes.py b/qubesbuilder/executors/qubes.py index 408577a5..e91dd8c3 100644 --- a/qubesbuilder/executors/qubes.py +++ b/qubesbuilder/executors/qubes.py @@ -22,10 +22,20 @@ import subprocess from pathlib import Path, PurePath from shlex import quote -from typing import List, Tuple, Union +from time import sleep +from typing import List, Optional, Tuple, Union from qubesbuilder.common import sanitize_line, PROJECT_PATH from qubesbuilder.executors import Executor, ExecutorError +from qubesbuilder.executors.qrexec import ( + create_dispvm, + kill_vm, + qrexec_call, + remove_vm, + start_vm, + vm_state, +) +from qubesbuilder.executors.windows import BaseWindowsExecutor # From https://github.com/QubesOS/qubes-core-admin-client/blob/main/qubesadmin/utils.py#L159-L173 @@ -65,6 +75,10 @@ def __init__(self, dispvm: str = "dom0", **kwargs): self._dispvm_template = "dom0" else: self._dispvm_template = dispvm + self.name = os.uname().nodename + self.dispvm: Optional[str] = None # actual dispvm name + self.copy_in_service = "qubesbuilder.FileCopyIn" + self.copy_out_service = "qubesbuilder.FileCopyOut" def get_user(self): return "user" @@ -72,36 +86,32 @@ def get_user(self): def get_group(self): return "user" - def copy_in(self, vm: str, source_path: Path, destination_dir: PurePath): # type: ignore + def copy_in(self, source_path: Path, destination_dir: PurePath, ignore_symlinks: bool = False): # type: ignore + assert self.dispvm src = source_path.expanduser().resolve() dst = destination_dir encoded_dst_path = encode_for_vmexec(str((dst / src.name).as_posix())) - copy_in_cmd = [ - "/usr/lib/qubes/qrexec-client-vm", - "--", - vm, - f"qubesbuilder.FileCopyIn+{encoded_dst_path}", - "/usr/lib/qubes/qfile-agent", - str(src), - ] - try: - self.log.debug(f"copy-in (cmd): {' '.join(copy_in_cmd)}") - subprocess.run(copy_in_cmd, check=True) - except subprocess.CalledProcessError as e: - if e.stderr is not None: - content = sanitize_line(e.stderr.rstrip(b"\n")).rstrip() - else: - content = str(e) - msg = f"Failed to copy-in: {content}" - raise ExecutorError(msg, name=vm) + + args = ["/usr/lib/qubes/qfile-agent"] + if ignore_symlinks: + args += ["--ignore-symlinks"] + args += [str(src)] + + qrexec_call( + log=self.log, + what="copy-in", + vm=self.dispvm, + service=f"{self.copy_in_service}+{encoded_dst_path}", + args=args, + ) def copy_out( self, - vm: str, source_path: PurePath, destination_dir: Path, dig_holes=False, ): # type: ignore + assert self.dispvm src = source_path dst = destination_dir.resolve() @@ -122,32 +132,53 @@ def copy_out( else: unpacker_path = old_unpacker_path encoded_src_path = encode_for_vmexec(str(src)) - cmd = [ - "/usr/lib/qubes/qrexec-client-vm", - vm, - f"qubesbuilder.FileCopyOut+{encoded_src_path}", - unpacker_path, - str(os.getuid()), - str(dst), - ] - try: - self.log.debug(f"copy-out (cmd): {' '.join(cmd)}") - subprocess.run(cmd, check=True) + qrexec_call( + log=self.log, + what="copy-out", + vm=self.dispvm, + service=f"{self.copy_out_service}+{encoded_src_path}", + args=[ + unpacker_path, + str(os.getuid()), + str(dst), + ], + ) - if dig_holes and not dst_path.is_dir(): + if dig_holes and not dst_path.is_dir(): + try: self.log.debug( "copy-out (detect zeroes and replace with holes)" ) subprocess.run( ["/usr/bin/fallocate", "-d", str(dst_path)], check=True ) - except subprocess.CalledProcessError as e: - if e.stderr is not None: - content = sanitize_line(e.stderr.rstrip(b"\n")).rstrip() - else: - content = str(e) - msg = f"Failed to copy-out: {content}" - raise ExecutorError(msg, name=vm) + except subprocess.CalledProcessError as e: + if e.stderr is not None: + content = sanitize_line(e.stderr.rstrip(b"\n")).rstrip() + else: + content = str(e) + msg = f"Failed to dig holes in copy-out: {content}" + raise ExecutorError(msg, name=self.dispvm) + + def copy_rpc_services(self): + assert self.dispvm + qrexec_call( + log=self.log, + what="copy builder rpc services", + vm=self.dispvm, + service="qubes.Filecopy", + args=[ + "/usr/lib/qubes/qfile-agent", + str(PROJECT_PATH / "rpc" / self.copy_in_service), + str(PROJECT_PATH / "rpc" / self.copy_out_service), + ], + options=["--filter-escape-chars-stderr"], + ) + + def cleanup(self): + # Kill the DispVM to prevent hanging for while + assert self.dispvm + kill_vm(self.log, self.dispvm) class LinuxQubesExecutor(QubesExecutor): @@ -156,15 +187,6 @@ def __init__( ): super().__init__(dispvm=dispvm, clean=clean, **kwargs) - @staticmethod - def cleanup(dispvm): - # Kill the DispVM to prevent hanging for while - subprocess.run( - ["qrexec-client-vm", "--", dispvm, "admin.vm.Kill"], - stdin=subprocess.DEVNULL, - stdout=subprocess.DEVNULL, - ) - def run( # type: ignore self, cmd: List[str], @@ -175,62 +197,14 @@ def run( # type: ignore no_fail_copy_out_allowed_patterns=None, dig_holes: bool = False, ): - dispvm = None try: - result = subprocess.run( - [ - "qrexec-client-vm", - "--", - self._dispvm_template, - "admin.vm.CreateDisposable", - ], - capture_output=True, - stdin=subprocess.DEVNULL, - ) - stdout = result.stdout - if not stdout.startswith(b"0\x00"): - raise ExecutorError("Failed to create disposable qube.") - stdout = stdout[2:] - if not re.match(rb"\Adisp(0|[1-9][0-9]{0,8})\Z", stdout): - raise ExecutorError("Failed to create disposable qube.") - try: - dispvm = stdout.decode("ascii", "strict") - except UnicodeDecodeError as e: - raise ExecutorError( - f"Failed to obtain disposable qube name: {str(e)}" - ) - - # Start the DispVM - subprocess.run( - [ - "/usr/lib/qubes/qrexec-client-vm", - dispvm, - "admin.vm.Start", - ], - stdin=subprocess.DEVNULL, - capture_output=True, - check=True, - ) - - # Copy qubes-builder RPC - copy_rpc_cmd = [ - "/usr/lib/qubes/qrexec-client-vm", - "--filter-escape-chars-stderr", - dispvm, - "qubes.Filecopy", - "/usr/lib/qubes/qfile-agent", - str(PROJECT_PATH / "rpc" / "qubesbuilder.FileCopyIn"), - str(PROJECT_PATH / "rpc" / "qubesbuilder.FileCopyOut"), - ] - subprocess.run( - copy_rpc_cmd, - stdin=subprocess.DEVNULL, - capture_output=True, - check=True, - ) + self.dispvm = create_dispvm(self.log, self._dispvm_template) + start_vm(self.log, self.dispvm) + self.copy_rpc_services() + assert self.dispvm prep_cmd = build_run_cmd_and_list( - dispvm, + self.dispvm, [ [ "sudo", @@ -248,8 +222,8 @@ def run( # type: ignore "mv", "-f", "--", - f"/home/{self.get_user()}/QubesIncoming/{os.uname().nodename}/qubesbuilder.FileCopyIn", - f"/home/{self.get_user()}/QubesIncoming/{os.uname().nodename}/qubesbuilder.FileCopyOut", + f"/home/{self.get_user()}/QubesIncoming/{self.name}/qubesbuilder.FileCopyIn", + f"/home/{self.get_user()}/QubesIncoming/{self.name}/qubesbuilder.FileCopyOut", "/usr/local/etc/qubes-rpc/", ], [ @@ -282,7 +256,7 @@ def run( # type: ignore for src_in, dst_in in sorted( set(copy_in or []), key=lambda x: x[1] ): - self.copy_in(dispvm, source_path=src_in, destination_dir=dst_in) + self.copy_in(source_path=src_in, destination_dir=dst_in) # replace placeholders if files_inside_executor_with_placeholders and isinstance( @@ -304,7 +278,7 @@ def run( # type: ignore .replace("\n", "\\\n") ) sed_cmd = build_run_cmd( - dispvm, + self.dispvm, [ "sed", "-i", @@ -325,7 +299,7 @@ def run( # type: ignore bash_env.append(f"{str(key)}={str(val)}") qvm_run_cmd = build_run_cmd( - dispvm, + self.dispvm, [ "env", "--", @@ -336,14 +310,14 @@ def run( # type: ignore ], ) - self.log.info(f"Using executor qubes:{dispvm}.") + self.log.info(f"Using executor qubes:{self.dispvm}.") self.log.debug(" ".join(qvm_run_cmd)) # stream output for command rc = self.execute(qvm_run_cmd) if rc != 0: msg = f"Failed to run '{' '.join(qvm_run_cmd)}' (status={rc})." - raise ExecutorError(msg, name=dispvm) + raise ExecutorError(msg, name=self.dispvm) # copy-out hook for src_out, dst_out in sorted( @@ -351,7 +325,6 @@ def run( # type: ignore ): try: self.copy_out( - dispvm, source_path=src_out, destination_dir=dst_out, dig_holes=dig_holes, @@ -372,9 +345,139 @@ def run( # type: ignore continue raise e except (subprocess.CalledProcessError, ExecutorError) as e: - if dispvm and self._clean_on_error: - self.cleanup(dispvm) + if self.dispvm and self._clean_on_error: + self.cleanup() raise e else: - if dispvm and self._clean: - self.cleanup(dispvm) + if self.dispvm and self._clean: + self.cleanup() + + +class WindowsQubesExecutor(BaseWindowsExecutor, QubesExecutor): + def __init__( + self, + ewdk: str, + dispvm: str = "win-build", + user: str = "user", + clean: Union[str, bool] = True, + **kwargs, + ): + super().__init__( + ewdk=ewdk, dispvm=dispvm, user=user, clean=clean, **kwargs + ) + self.copy_in_service = "qubesbuilder.WinFileCopyIn" + self.copy_out_service = "qubesbuilder.WinFileCopyOut" + + def start_worker(self): + # we need the dispvm in a stopped state to attach EWDK block device + self.dispvm = create_dispvm(self.log, self._dispvm_template) + self.log.debug(f"dispvm: {self.dispvm}") + self.attach_ewdk(self.dispvm) + start_vm(self.log, self.dispvm) + + # wait for startup + for _ in range(10): + try: + qrexec_call( + log=self.log, + what="dispvm qrexec test", + vm=self.dispvm, + service="qubes.VMShell", + stdin=b"exit 0", + ) + return # all good + except ExecutorError as e: + self.log.debug(f"VMShell failed: {e}") + sleep(5) + raise ExecutorError( + f"Failed to communicate with windows dispvm '{self.dispvm}'" + ) + + def cleanup(self): + if self.dispvm is None: + return + + state = vm_state(self.log, self.dispvm) + + if state != "Halted": + kill_vm(self.log, self.dispvm) + else: + remove_vm(self.log, self.dispvm) + + def run( + self, + cmd: List[str], + copy_in: List[Tuple[Path, PurePath]] = None, + copy_out: List[Tuple[PurePath, Path]] = None, + ): + try: + # copy the rpc handlers + self.start_worker() + # TODO: don't require two scripts per service + assert self.dispvm + qrexec_call( + log=self.log, + what="copy RPC services to dispvm", + vm=self.dispvm, + service="qubes.Filecopy", + args=[ + "/usr/lib/qubes/qfile-agent", + str(PROJECT_PATH / "rpc" / "qubesbuilder.WinFileCopyIn"), + str(PROJECT_PATH / "rpc" / "qubesbuilder.WinFileCopyOut"), + str(PROJECT_PATH / "rpc" / "qubesbuilder-file-copy-in.ps1"), + str( + PROJECT_PATH / "rpc" / "qubesbuilder-file-copy-out.ps1" + ), + ], + ) + + inc_dir = ( + f"c:\\users\\{self.user}\\Documents\\QubesIncoming\\{self.name}" + ) + + prep_cmd = [ + f'move /y "{inc_dir}\\qubesbuilder.WinFileCopyIn" "%QUBES_TOOLS%\\qubes-rpc\\"', + f'move /y "{inc_dir}\\qubesbuilder.WinFileCopyOut" "%QUBES_TOOLS%\\qubes-rpc\\"', + f'move /y "{inc_dir}\\qubesbuilder-file-copy-in.ps1" "%QUBES_TOOLS%\\qubes-rpc-services\\"', + f'move /y "{inc_dir}\\qubesbuilder-file-copy-out.ps1" "%QUBES_TOOLS%\\qubes-rpc-services\\"', + ] + + qrexec_call( + log=self.log, + what="prepare RPC services in dispvm", + vm=self.dispvm, + service="qubes.VMShell", + stdin=( + " & ".join(prep_cmd) + " & exit !errorlevel!" + "\r\n" + ).encode("utf-8"), + ) + + for src_in, dst_in in copy_in or []: + self.copy_in(src_in, dst_in, ignore_symlinks=True) + + bin_cmd = ( + " & ".join(cmd) + " & exit !errorlevel!" + "\r\n" + ).encode("utf-8") + self.log.debug(f"{bin_cmd=}") + + # TODO: this doesn't stream stdout, use execute() + stdout = qrexec_call( + log=self.log, + what="run command in dispvm", + vm=self.dispvm, + service="qubes.VMShell", + stdin=bin_cmd, + capture_output=True, + ) + assert stdout is not None + self.log.debug(stdout.decode("utf-8")) + + for src_out, dst_out in copy_out or []: + self.copy_out(src_out, dst_out) + except ExecutorError as e: + suffix = f" in qube {self.dispvm}" if self.dispvm else "" + raise ExecutorError( + f"Failed to run command{suffix}: {str(e)}" + ) from e + finally: + self.cleanup() diff --git a/qubesbuilder/executors/windows.py b/qubesbuilder/executors/windows.py new file mode 100644 index 00000000..4a51203b --- /dev/null +++ b/qubesbuilder/executors/windows.py @@ -0,0 +1,289 @@ +# The Qubes OS Project, http://www.qubes-os.org +# +# Copyright (C) 2021 Frédéric Pierret (fepitre) +# Copyright (C) 2024 Rafał Wojdyła +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# SPDX-License-Identifier: GPL-3.0-or-later + +import os +import subprocess +from abc import ABC +from pathlib import Path, PurePath, PureWindowsPath +from time import sleep +from typing import List, Optional, Tuple + +from qubesbuilder.common import sanitize_line +from qubesbuilder.executors import Executor, ExecutorError +from qubesbuilder.executors.qrexec import ( + create_dispvm, + kill_vm as qkill_vm, + qrexec_call, + start_vm, + vm_state, +) + + +class BaseWindowsExecutor(Executor, ABC): + def __init__( + self, ewdk: Optional[str] = None, user: str = "user", **kwargs + ): + super().__init__(**kwargs) + self.ewdk_path = ewdk + self.user = user + + def get_builder_dir(self): + return PureWindowsPath("c:\\builder") + + def get_user(self): + return self.user + + def get_threads(self) -> int: + return self._kwargs.get("threads", 1) + + # starts dispvm from default template (not windows) + def start_dispvm(self) -> str: + name = create_dispvm(self.log, "dom0") + self.log.debug(f"created dispvm {name}") + start_vm(self.log, name) + return name + + def kill_vm(self, vm: str): + qkill_vm(self.log, vm) + + def run_rpc_service( + self, + target: str, + service: str, + description: str, + stdin: bytes = b"", + ) -> bytes: + out = qrexec_call( + log=self.log, + what=description, + vm=target, + service=service, + capture_output=True, + stdin=stdin, + ) + assert out is not None + return out + + # Get loop device id for the EWDK iso if attached to the builder vm, otherwise None + def _get_ewdk_loop(self) -> Optional[str]: + if self.ewdk_path is None: + return None + + try: + proc = subprocess.run( + ["losetup", "-j", self.ewdk_path], + check=True, + capture_output=True, + ) + stdout = proc.stdout.decode() + if "/dev/loop" in stdout: + loop_dev = stdout.split(":", 1)[0] + loop_id = loop_dev.removeprefix("/dev/") + self.log.debug(f"ewdk loop id: {loop_id}") + return loop_id + else: + return None + + except subprocess.CalledProcessError as e: + raise ExecutorError( + f"Failed to run losetup: {proc.stderr.decode()}" + ) from e + + def attach_ewdk(self, vm: str): + assert self.ewdk_path + if not Path(self.ewdk_path).is_file(): + raise ExecutorError(f"EWDK image not found at '{self.ewdk_path}'") + + loop_id = self._get_ewdk_loop() + if not loop_id: + # attach EWDK image + proc = None + try: + self.log.debug(f"attaching EWDK from '{self.ewdk_path}'") + proc = subprocess.run( + ["sudo", "losetup", "-f", self.ewdk_path], + check=True, + capture_output=True, + ) + except subprocess.CalledProcessError as e: + raise ExecutorError( + f"Failed to run losetup: {proc.stderr.decode() if proc else e}" + ) from e + + loop_id = self._get_ewdk_loop() + if not loop_id: + raise ExecutorError(f"Failed to attach EWDK ({self.ewdk_path})") + + self.log.debug(f"attached EWDK as '{loop_id}'") + + # wait for device to appear + self_name = os.uname().nodename + timeout = 10 + while timeout > 0: + stdout = qrexec_call( + log=self.log, + what="ewdk loop device query", + vm=self_name, + service=f"admin.vm.device.block.Available+{loop_id}", + ) + + assert stdout is not None + if stdout.decode("ascii", "strict").startswith(loop_id): + # loop device ready, attach to vm + qrexec_call( + log=self.log, + what="attach ewdk to worker vm", + vm=vm, + service=f"admin.vm.device.block.Attach+{self_name}+{loop_id}", + stdin=b"devtype=cdrom read-only=true persistent=true", + ) + return + + timeout -= 1 + sleep(1) + + raise ExecutorError( + f"Failed to attach EWDK ({self.ewdk_path}): " + f"wait for loopback device timed out" + ) + + +class SSHWindowsExecutor(BaseWindowsExecutor): + def __init__( + self, + ewdk: str, + ssh_ip: str, + ssh_key_path: str = "/home/user/.ssh/win-build.key", + user: str = "user", + ssh_vm: Optional[str] = None, + **kwargs, + ): + super().__init__(ewdk=ewdk, user=user, **kwargs) + self.ip = ssh_ip + self.key_path = ssh_key_path + self.vm = ssh_vm + + def ssh_cmd(self, cmd: List[str]): + target = f"{self.user}@{self.ip}" + ret = self.execute( + cmd=[ + "ssh", + "-i", + self.key_path, + "-o", + "BatchMode yes", + "-o", + "StrictHostKeyChecking accept-new", + "-o", + "ConnectTimeout 60", + target, + "cmd", + "/e", + "/v:on", + "/c", + " & ".join(cmd), + " & exit !errorlevel!", + ], + ) + if ret != 0: + raise ExecutorError(f"Failed to run SSH cmd {cmd}") + + def copy_in(self, source_path: Path, destination_dir: PurePath): + src = str(source_path.expanduser().resolve()) + dst = str(destination_dir) + self.log.debug(f"copy_in: {src} -> {dst}") + + self.ssh_cmd( + [ + f'if not exist "{dst}" md "{dst}"', + ] + ) + + target = f"{self.user}@{self.ip}" + self.execute( + [ + "scp", + "-i", + self.key_path, + "-r", + "-B", + "-q", + src, + f"{target}:{dst}", + ] + ) + + def copy_out(self, source_path: PurePath, destination_dir: Path): + self.log.debug(f"copy_out: {source_path} -> {destination_dir}") + src = str(source_path) + dst = str(destination_dir.expanduser().resolve()) + + target = f"{self.user}@{self.ip}" + self.execute( + [ + "scp", + "-i", + self.key_path, + "-r", + "-B", + "-q", + f"{target}:{src.replace('\\', '/')}", + dst, + ] + ) + + def start_worker(self): + assert self.vm is not None + # we need the vm in a stopped state to attach EWDK block device + self.log.debug(f"starting worker vm: {self.vm}") + + try: + self.attach_ewdk(self.vm) + except ExecutorError as e: + if "DeviceAlreadyAttached" in str(e): + self.log.debug(f"EWDK already attached to vm '{self.vm}'") + + start_vm(self.log, self.vm) + # ensure connectivity + self.ssh_cmd(["exit 0"]) + + def run( + self, + cmd: List[str], + copy_in: List[Tuple[Path, PurePath]] = None, + copy_out: List[Tuple[PurePath, Path]] = None, + ): + if self.vm is not None: + self.start_worker() + + # this executor doesn't use a dispvm, clear the build dir every time + self.ssh_cmd( + [ + f'if exist "{self.get_builder_dir()}" rmdir /s /q "{self.get_builder_dir()}"' + ] + ) + + for src_in, dst_in in copy_in or []: + self.copy_in(src_in, dst_in) + + self.ssh_cmd(cmd) + + for src_out, dst_out in copy_out or []: + self.copy_out(src_out, dst_out) diff --git a/qubesbuilder/plugins/__init__.py b/qubesbuilder/plugins/__init__.py index 8697d319..2cfe553b 100644 --- a/qubesbuilder/plugins/__init__.py +++ b/qubesbuilder/plugins/__init__.py @@ -437,6 +437,12 @@ def from_args(cls, **kwargs): return cls(**kwargs) +def get_stage_options(stage: str, options: dict): + stages = options.get("stages", []) + s: dict = next((s for s in stages if stage in s), {}) + return s.get(stage, {}) + + class DistributionComponentPlugin(DistributionPlugin, ComponentPlugin): @classmethod def from_args(cls, **kwargs): @@ -504,6 +510,12 @@ def update_parameters(self, stage: str): f"{self.component}:{self.dist}: Conflicting build paths" ) + def get_config_stage_options(self, stage: str): + stage_options = {} + stage_options.update(get_stage_options(stage, self.dist.kwargs)) + stage_options.update(get_stage_options(stage, self.component.kwargs)) + return stage_options + def get_dist_component_artifacts_dir_history( self, stage: str ) -> List[Path]: @@ -672,3 +684,9 @@ class GentooDistributionPlugin(DistributionPlugin): @classmethod def supported_distribution(cls, distribution: QubesDistribution): return distribution.is_gentoo() + + +class WindowsDistributionPlugin(DistributionPlugin): + @classmethod + def supported_distribution(cls, distribution: QubesDistribution): + return distribution.is_windows() diff --git a/qubesbuilder/plugins/build_windows/__init__.py b/qubesbuilder/plugins/build_windows/__init__.py new file mode 100644 index 00000000..ed30bfc3 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/__init__.py @@ -0,0 +1,522 @@ +# The Qubes OS Project, http://www.qubes-os.org +# +# Copyright (C) 2021 Frédéric Pierret (fepitre) +# Copyright (C) 2024 Rafał Wojdyła +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# SPDX-License-Identifier: GPL-3.0-or-later +import logging +import os.path +import re +import shutil +import subprocess +import yaml +from enum import StrEnum +from pathlib import Path +from typing import Dict, ItemsView, List, Optional + +from qubesbuilder.component import QubesComponent +from qubesbuilder.config import Config +from qubesbuilder.distribution import QubesDistribution +from qubesbuilder.executors import ExecutorError +from qubesbuilder.executors.windows import BaseWindowsExecutor +from qubesbuilder.plugins import WindowsDistributionPlugin, PluginDependency +from qubesbuilder.plugins.build import BuildPlugin, BuildError + + +class WinArtifactKind(StrEnum): + BIN = "bin" + INC = "inc" + LIB = "lib" + + def __repr__(self) -> str: + return self.name + + +yaml.SafeDumper.add_representer( + WinArtifactKind, yaml.representer.SafeRepresenter.represent_str +) + + +class WinArtifactSet: + def __init__(self, artifacts: Dict[WinArtifactKind, List[str]] = None): + self.artifacts = artifacts or {kind: [] for kind in WinArtifactKind} + + def __repr__(self) -> str: + return self.items().__repr__() + + def items(self) -> ItemsView[WinArtifactKind, List[str]]: + return self.artifacts.items() + + def add(self, kind: WinArtifactKind, file: str): + self.artifacts[kind] += [file] + + def get_kind(self, kind: WinArtifactKind) -> List[str]: + return self.artifacts[kind] + + +def clean_local_repository( + log: logging.Logger, + repository_dir: Path, + component: QubesComponent, + dist: QubesDistribution, + all_versions: bool = False, +): + """ + Remove component from local repository. + """ + log.info( + f"{component}:{dist}: Cleaning local repository '{repository_dir}'" + f"{' (all versions)' if all_versions else ''}." + ) + if all_versions: + for version_dir in repository_dir.glob(f"{component.name}_*"): + shutil.rmtree(version_dir.as_posix()) + else: + target_dir = repository_dir / f"{component.name}_{component.version}" + if target_dir.exists(): + shutil.rmtree(target_dir.as_posix()) + + +def provision_local_repository( + log: logging.Logger, + repository_dir: Path, + component: QubesComponent, + dist: QubesDistribution, + target: str, + artifacts: WinArtifactSet, + build_artifacts_dir: Path, + test_sign: bool, +): + """ + Provision local builder repository. + """ + log.info( + f"{component}:{dist}:{target}: Provisioning local repository '{repository_dir}'." + ) + + target_dir = repository_dir / f"{component.name}_{component.version}" + target_dir.mkdir(parents=True, exist_ok=True) + + try: + for kind in WinArtifactKind: + (target_dir / kind).mkdir(parents=True, exist_ok=True) + for file in artifacts.get_kind(kind): + pkg_path = build_artifacts_dir / kind / file + target_path = target_dir / kind / file + os.link(pkg_path, target_path) + if test_sign: + target_path = target_dir / "sign.crt" + src_path = build_artifacts_dir / "sign.crt" + if os.path.isfile( + src_path + ): # this may be absent for targets with no binary artifacts + os.link(src_path, target_path) + + except ( + ValueError, + PermissionError, + NotImplementedError, + FileExistsError, + ) as e: + msg = f"{component}:{dist}:{target}: Failed to provision local repository." + raise BuildError(msg) from e + + +def mangle_key_name(key_name: str) -> str: + return key_name.replace(" ", "__") + + +class WindowsBuildPlugin(WindowsDistributionPlugin, BuildPlugin): + """ + WindowsBuildPlugin manages Windows distribution build. + + Stages: + - build - Build VS solutions and provision local repository. + + Entry points: + - build + """ + + name = "build_windows" + stages = ["build"] + dependencies = [PluginDependency("build")] + + def __init__( + self, + component: QubesComponent, + dist: QubesDistribution, + config: Config, + stage: str, + **kwargs, + ): + super().__init__( + component=component, dist=dist, config=config, stage=stage + ) + + def update_parameters(self, stage: str): + super().update_parameters(stage) + + # Set and update parameters based on top-level "source", + # per package set and per distribution + parameters = self.component.get_parameters(self.get_placeholders(stage)) + self._parameters.update( + parameters.get(self.dist.package_set, {}).get("source", {}) + ) + self._parameters.update( + parameters.get(self.dist.distribution, {}).get("source", {}) + ) + + def update_placeholders(self, stage: str): + super().update_placeholders(stage) + stage_options = self.get_config_stage_options(stage) + self._placeholders[stage].update( + { + "@CONFIGURATION@": stage_options.get( + "configuration", "Release" + ), + } + ) + + # Generate self-signed key if test-signing, get public cert + def sign_prep( + self, + qube: str, + key_name: str, + test_sign: bool, + ) -> bytes: + if test_sign: + self.executor.run_rpc_service( + target=qube, + service=f"qubesbuilder.WinSign.CreateKey+{mangle_key_name(key_name)}", + description=f"create signing key '{key_name}'", + ) + + return self.executor.run_rpc_service( + target=qube, + service=f"qubesbuilder.WinSign.GetCert+{mangle_key_name(key_name)}", + description=f"get certificate for signing key '{key_name}'", + ) + + # Sign a file and return signed bytes + def sign_sign( + self, + qube: str, + key_name: str, + file: Path, + ) -> bytes: + with open(file, "rb") as f: + return self.executor.run_rpc_service( + target=qube, + service=f"qubesbuilder.WinSign.Sign+{mangle_key_name(key_name)}", + description=f"authenticode sign '{file}' with key '{key_name}'", + stdin=f.read(), + ) + + # Delete signing key + def sign_delete_key(self, qube: str, key_name: str): + out = self.executor.run_rpc_service( + target=qube, + service=f"qubesbuilder.WinSign.QueryKey+{mangle_key_name(key_name)}", + description=f"query signing key '{key_name}'", + ) + + if f"Key '{mangle_key_name(key_name)}' exists" not in out.decode( + "utf-8" + ): + self.log.debug(f"key '{key_name}' does not exist") + return + + self.executor.run_rpc_service( + target=qube, + service=f"qubesbuilder.WinSign.DeleteKey+{mangle_key_name(key_name)}", + description=f"delete signing key '{key_name}'", + ) + + def run(self): + """ + Run plugin for given stage. + """ + # Run stage defined by parent class + super().run() + self.log.debug(f"run start for {self.component.name}") + + if self.stage != "build" or not self.has_component_packages("build"): + return + + if not isinstance(self.executor, BaseWindowsExecutor): + raise BuildError( + f"Plugin {self.name} requires BaseWindowsExecutor, got {self.executor.__class__.__name__}" + ) + + parameters = self.get_parameters(self.stage) + distfiles_dir = self.get_component_distfiles_dir() + artifacts_dir = self.get_dist_component_artifacts_dir(self.stage) + + self.log.debug(f"{parameters=}") + stage_options = self.get_config_stage_options(self.stage) + self.log.debug(f"{stage_options=}") + + # Compare previous artifacts hash with current source hash + hash = self.get_dist_artifacts_info( + self.stage, self.component.name + ).get("source-hash", None) + if self.component.get_source_hash() == hash: + self.log.info( + f"{self.component}:{self.dist}: Source hash is the same than already built source. Skipping." + ) + return + + # Clean previous build artifacts + if artifacts_dir.exists(): + shutil.rmtree(artifacts_dir.as_posix()) + artifacts_dir.mkdir(parents=True) + + # Create output folders + output_dirs = { + kind.value: artifacts_dir / kind.value for kind in WinArtifactKind + } + for dir in output_dirs.values(): + dir.mkdir(parents=True) + + # Source artifacts + prep_artifacts_dir = self.get_dist_component_artifacts_dir(stage="prep") + + # Local build repository + repository_dir = self.config.repository_dir / self.dist.distribution + repository_dir.mkdir(parents=True, exist_ok=True) + + # Remove previous versions in order to keep the latest one only + clean_local_repository( + self.log, repository_dir, self.component, self.dist, True + ) + + builder_dir = str(self.executor.get_builder_dir()) + artifacts = WinArtifactSet() + + # Read information from source stage + source_info = self.get_dist_artifacts_info( + stage="prep", basename=self.component.name + ) + + # Authenticode signing prep + test_sign = stage_options.get("test-sign", True) + sign_qube = stage_options.get("sign-qube") + if not sign_qube: + raise BuildError("'sign-qube' option not configured") + sign_key_name = stage_options.get( + "sign-key-name", "Qubes Windows Tools" + ) + + dvm: Optional[str] = None + try: + for target in parameters["build"]: + self.log.debug(f"building {target}") + + # TODO: better mark that there's no target + do_build = str(target) != "dummy" + if do_build and target.suffix != ".sln": + raise BuildError( + f"Plugin {self.name} can only build Visual Studio .sln targets" + ) + + # Copy-in distfiles, source and dependencies repository + copy_in = self.default_copy_in( + self.executor.get_plugins_dir(), + self.executor.get_sources_dir(), + ) + copy_in += [ + ( + repository_dir, + self.executor.get_repository_dir(), + ), # deps + (self.component.source_dir, self.executor.get_build_dir()), + (distfiles_dir, self.executor.get_distfiles_dir()), + ] + + copy_out = [] + copy_out_dir = self.executor.get_builder_dir() / "copy_out" + copy_out_prep_cmds = [f'mkdir "{str(copy_out_dir)}"'] + + # Parse output files + for kind, dir in output_dirs.items(): + files = parameters.get(kind, []) + kind_dir = copy_out_dir / kind + copy_out_prep_cmds += [f'mkdir "{str(kind_dir)}"'] + copy_out += [(kind_dir, artifacts_dir)] + for file in files: + copy_out_prep_cmds += [ + f'copy "{str(self.executor.get_build_dir() / self.component.name / file)}" "{str(kind_dir)}"' + ] + artifacts.add(WinArtifactKind(kind), Path(file).name) + + if do_build: + cmds = [] + # create component-local link to distfiles, local build compatibility + cmd = [ + "mklink", + "/d", + str( + self.executor.get_build_dir() + / self.component.name + / ".distfiles" + ), + str( + self.executor.get_distfiles_dir() + / self.component.name + ), + ] + cmds += [" ".join(cmd)] + + cmd = [ + "powershell", + "-noninteractive", + "-executionpolicy", + "bypass", + f"{ self.executor.get_plugins_dir() / self.name / 'scripts' / 'build-sln.ps1' }", + "-solution", + str( + self.executor.get_build_dir() + / self.component.name + / target + ), + "-repo", + str( + self.executor.get_repository_dir() + / self.dist.distribution + ), + "-distfiles", + str( + self.executor.get_distfiles_dir() + / self.component.name + ), + ] + + cmd += [ + "-configuration", + self._placeholders[self.stage]["@CONFIGURATION@"], + ] + + if test_sign: + cmd += ["-testsign"] + + if self.config.debug: + cmd += ["-log"] # generate msbuild log + + if self.config.verbose: + cmd += ["-noisy"] + + if self.executor.get_threads() > 1: + cmd += ["-threads", str(self.executor.get_threads())] + cmds += [" ".join(cmd)] + cmds += copy_out_prep_cmds + else: # dummy + cmds = copy_out_prep_cmds + + # TODO: failed builds don't get caught here due to msbuild/powershell weirdness + # see scripts/build-sln.ps1 + # this is only a problem if a target has no outputs (otherwise copy_out fails) + try: + self.executor.run( + cmds, + copy_in, + copy_out, + ) + except ExecutorError as e: + msg = f"{self.component}:{self.dist}:{target}: Failed to build solution: {str(e)}." + raise BuildError(msg) from e + + if do_build: + # authenticode sign the binaries + if not os.path.isfile(artifacts_dir / "sign.crt"): + # get signing cert + sign_cert = self.sign_prep( + qube=sign_qube, + key_name=sign_key_name, + test_sign=test_sign, + ) + + with open(artifacts_dir / "sign.crt", "wb") as f: + f.write(sign_cert) + + # dvm is required for timestamping (need internet access) + dvm = self.executor.start_dispvm() + skip_test_sign = parameters.get("skip-test-sign", []) + for file in artifacts.get_kind(WinArtifactKind.BIN): + if not Path(file).suffix in [ + ".cat", + ".dll", + ".exe", + ".sys", + ]: + continue + + if test_sign and file in skip_test_sign: + continue + + path = artifacts_dir / "bin" / file + signed_data = self.sign_sign( + qube=sign_qube, + key_name=sign_key_name, + file=path, + ) + + io = self.executor.run_rpc_service( + target=dvm, + service="qubesbuilder.WinSign.Timestamp", + description=f"authenticode timestamp {file}", + stdin=signed_data, + ) + + # TODO: should we keep unsigned binaries? + signed_path = str(path) + ".signed" + with open(signed_path, "wb") as f: + f.write(io) + + os.replace(signed_path, path) + + provision_local_repository( + log=self.log, + repository_dir=repository_dir, + component=self.component, + dist=self.dist, + target=target, + artifacts=artifacts, + build_artifacts_dir=artifacts_dir, + test_sign=test_sign, + ) + + info = source_info + info.update( + { + "files": artifacts.artifacts, + "source-hash": self.component.get_source_hash(), + } + ) + self.save_dist_artifacts_info( + stage=self.stage, basename=self.component.name, info=info + ) + finally: + if test_sign: + self.sign_delete_key( + qube=sign_qube, + key_name=sign_key_name, + ) + + if dvm: + self.executor.kill_vm(dvm) + + +PLUGINS = [WindowsBuildPlugin] diff --git a/qubesbuilder/plugins/build_windows/scripts/build-sln.ps1 b/qubesbuilder/plugins/build_windows/scripts/build-sln.ps1 new file mode 100755 index 00000000..28202de1 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/build-sln.ps1 @@ -0,0 +1,135 @@ +# The Qubes OS Project, http://www.qubes-os.org +# +# Copyright (C) 2024 Rafał Wojdyła +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# SPDX-License-Identifier: GPL-3.0-or-later + +# Generic script for building VS solutions using EWDK + +param( + [Parameter(Mandatory=$true)] [string]$solution, + # root of local builder repository with dependencies, sets QUBES_REPO env variable + [Parameter(Mandatory=$true)] [string]$repo, + # directory with distfiles (additional downloaded source files), sets QUBES_DISTFILES env variable + [string]$distfiles = "", + [string]$configuration = "Release", + [int]$threads = 1, + [switch]$testsign = $false, # used to set TEST_SIGN env variable so the installer can bundle public certs + [switch]$noisy = $false, + [switch]$log = $false +) + +$ErrorActionPreference = "Stop" + +$arch = "x64" +$log_file = (Split-Path -Parent -Resolve $solution) + "\msbuild.binlog" + +. $PSScriptRoot\common.ps1 + +LogStart +LogInfo "Building $solution, $configuration, $arch, $threads thread(s)" + +$ewdk = Find-EWDK + +if ($ewdk -eq $null) { + LogError "EWDK not found. If it's not attached as a drive, set its location in the EWDK_PATH environment variable." +} + +$env:EWDK_PATH = $ewdk +$msbuild = "$env:EWDK_PATH\Program Files\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\MSBuild.exe" +if (! (Test-Path -Path $msbuild)) { + LogError "$msbuild not found." +} + +# Prepare environment for build +Launch-EWDK + +$ewdk_version = $env:Version_Number +if ($ewdk_version -eq $null) { + LogError "EWDK environment initialization failed." +} + +$ewdk_inc_dir = "$env:EWDK_PATH\Program Files\Windows Kits\10\Include\$ewdk_version" +$ewdk_inc = "$ewdk_inc_dir\shared;$ewdk_inc_dir\um;$ewdk_inc_dir\ucrt" + +$ewdk_lib_dir = "$env:EWDK_PATH\Program Files\Windows Kits\10\Lib\$ewdk_version" +$ewdk_lib = "$ewdk_lib_dir\um\$arch;$ewdk_lib_dir\ucrt\$arch" + +$env:WindowsSDK_IncludePath = $ewdk_inc +Set-Item -Path "env:WindowsSDK_LibraryPath_$arch" -Value $ewdk_lib +$env:PATH += ";$env:EWDK_PATH\Program Files\Windows Kits\10\bin\$ewdk_version\$arch" + +$build_args = @("$solution", "-restore", "-t:Rebuild", "-p:Platform=$arch", "-p:Configuration=$configuration", "-m:$threads", "-nologo") + +if (! $noisy) { + $build_args += "-v:quiet" +} + +if ($log) { + $build_args += "-bl:$log_file" +} + +if ($testsign) { + $env:TEST_SIGN = 1 +} + +# Iterate over builder's local repository to collect dependencies +if (! (Test-Path $repo -PathType Container)) { + LogError "Invalid repository directory: $repo" +} + +$env:QUBES_REPO = Resolve-Path $repo +foreach ($dep in Get-ChildItem -Path $repo) { + # strip version numbers from directories so projects can use constant paths for deps + if ($dep.name.lastindexof('_') -ge 0) { + $new_dep = $dep.name.substring(0, $dep.name.lastindexof('_')) + mv "$repo\$dep" "$repo\$new_dep" + $dep = $new_dep + } + + $inc_path = "$repo\$dep\inc" + if (Test-Path -Path $inc_path) { + $env:QUBES_INCLUDES += ";$inc_path" + } + + $lib_path = "$repo\$dep\lib" + if (Test-Path -Path $lib_path) { + $env:QUBES_LIBS += ";$lib_path" + } +} + +LogDebug "QUBES_INCLUDES = $env:QUBES_INCLUDES" +LogDebug "QUBES_LIBS = $env:QUBES_LIBS" + +if ($distfiles -ne "") { + if (! (Test-Path $distfiles -PathType Container)) { + LogError "Invalid distfiles directory: $distfiles" + } + $env:QUBES_DISTFILES = Resolve-Path $distfiles + LogDebug "QUBES_DISTFILES = $env:QUBES_DISTFILES" +} else { + LogDebug "no distfiles" +} + +LogDebug "msbuild args: $build_args" + +# Start-Process -Wait hangs here for some reason, but waiting separately works properly +# seems to be related to msbuild leaving some worker processes running +# TODO: investigate, this doesn't return correct exit code if build fails +# ($proc.ExitCode is null?!) +$proc = Start-Process -FilePath $msbuild -NoNewWindow -PassThru -ArgumentList $build_args +$proc.WaitForExit() +return $proc.ExitCode diff --git a/qubesbuilder/plugins/build_windows/scripts/common.ps1 b/qubesbuilder/plugins/build_windows/scripts/common.ps1 new file mode 100755 index 00000000..f426d998 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/common.ps1 @@ -0,0 +1,82 @@ +# Get EWDK root from the environment or find it if not set +function Find-EWDK { + if (Test-Path -Path env:EWDK_PATH) { + return $env:EWDK_PATH + } else { + foreach ($drive in Get-PSDrive) { + if ($drive.Provider.Name -eq "FileSystem") { + $root = $drive.Root + $path = "$root\LaunchBuildEnv.cmd" + if (Test-Path -Path $path) { + return $root + } + } + } + } + return $null +} + +# Launch EWDK's environment setup script and grab variables that were set +function Launch-EWDK { + $ewdk_env_cmd = "$env:EWDK_PATH\BuildEnv\SetupBuildEnv.cmd" + $ewdk_vars_txt = cmd /c "$ewdk_env_cmd x86_amd64 > nul & set" + + foreach ($line in $ewdk_vars_txt) { + $kv = $line.split("=") + $var_name = $kv[0] + $var_value = $kv[1] + if (! (Test-Path -Path "env:$var_name")) { + Set-Item -Path "env:$var_name" -Value $var_value + } + } +} + +function LogStart { + $logDir = "c:\builder\log" + New-Item -Path $logDir -ItemType Directory -Force + $baseName = (Get-Item $MyInvocation.PSCommandPath).BaseName + $logname = "$baseName-$(Get-Date -Format "yyyyMMdd-HHmmss")-$PID.log" + $global:qwtLogPath = "$logDir\$logName" + $global:qwtLogLevel = 4 +} + +function Log { + param ( + [ValidateRange(1,5)][int]$level, + [string]$msg + ) + + if ($level -le $qwtLogLevel) { + $ts = Get-Date -Format "yyyyMMdd.HHmmss.fff" + Add-Content $qwtLogPath -value "[$ts-$("EWIDV"[$level-1])] $msg" + } +} + +function LogError { + param([string]$msg) + Log 1 $msg + Write-Error $msg +} + +function LogWarning { + param([string]$msg) + Log 2 $msg + Write-Warning $msg +} + +function LogInfo { + param([string]$msg) + Log 3 $msg + Write-Host $msg +} + +function LogDebug { + param([string]$msg) + Log 4 $msg + Write-Host $msg +} + +function LogVerbose { + param([string]$msg) + Log 5 $msg +} diff --git a/qubesbuilder/plugins/build_windows/scripts/local/build-qwt.ps1 b/qubesbuilder/plugins/build_windows/scripts/local/build-qwt.ps1 new file mode 100755 index 00000000..bdc61493 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/local/build-qwt.ps1 @@ -0,0 +1,45 @@ +# Build Qubes Windows Tools locally, with test signing. +# All build artifacts are copied to "repository" dir in current directory. +# The final installer is in repository/installer-windows-tools/bin. + +param( + [Parameter(Mandatory, HelpMessage="Directory containing all components' sources")] [string]$src, + [Parameter(HelpMessage="Build configuration (Release/Debug)")] [string]$cfg = "Release" +) + +$ErrorActionPreference = "Stop" + +# list of required components, in order of dependencies +# source directories can also have "qubes-" prepended to the name +$components = @( + "vmm-xen-windows-pvdrivers", + "core-vchan-xen", + "windows-utils", + "core-qubesdb", + "core-agent-windows", + "gui-common", + "gui-agent-windows", + "installer-windows-tools" +) + +if (! (Test-Path $src -PathType Container)) { + Write-Error "Invalid source directory: $src" +} + +$repo = ".\repository" + +if (Test-Path $repo) { + Remove-Item -Path $repo -Recurse -Force +} +New-Item -Path "$repo" -ItemType Directory -Force +$repo = Resolve-Path $repo + +foreach ($component in $components) { + if (! (Test-Path "$src\$component" -PathType Container)) { + $component = "qubes-" + $component + if (! (Test-Path "$src\$component" -PathType Container)) { + Write-Error "Component '$component' not found in directory '$src'" + } + } + & "$PSScriptRoot\build.ps1" "$src\$component" "$repo" $cfg +} diff --git a/qubesbuilder/plugins/build_windows/scripts/local/build.ps1 b/qubesbuilder/plugins/build_windows/scripts/local/build.ps1 new file mode 100755 index 00000000..2cdd9233 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/local/build.ps1 @@ -0,0 +1,130 @@ +# Build a Windows component locally, with test signing + +param( + [Parameter(Mandatory, HelpMessage="Component's source directory")] [string]$dir, + [Parameter(Mandatory, HelpMessage="Repository directory for build artifacts")] [string]$repo, + [Parameter(Mandatory, HelpMessage="Build configuration (Release/Debug)")] [string]$cfg = "Release" +) + +$ErrorActionPreference = "Stop" + +if (! (Test-Path $dir -PathType Container)) { + Write-Error "Invalid component directory: $dir" +} + +$dir = Resolve-Path $dir + +$script_dir = Resolve-Path "$PSScriptRoot\.." + +# create local repo, without version since it's stripped by the build script anyway +$component = Split-Path $dir -Leaf +$repo_component = $component.TrimStart("qubes-") # normalize name +$repo_dir = "$repo\$repo_component" +if (Test-Path $repo_dir) { + Remove-Item -Path $repo_dir -Recurse -Force +} +New-Item -Path $repo_dir -ItemType Directory -Force + +$component_version = (Get-Content "$dir\version").Trim() + +function ReplacePlaceholders { + param([string]$str) + return $str.Replace("@CONFIGURATION@", $cfg).Replace("@VERSION@", $component_version) +} + +Import-Module powershell-yaml + +$yaml = ConvertFrom-Yaml (Get-Content "$dir\.qubesbuilder" -Raw) + +# download distfiles +if ($yaml.ContainsKey('source') -and $yaml['source'].ContainsKey('files')) { + # for local builds, keep distfiles in the source dir for easy access by the component + $distfiles = "$dir\.distfiles" + New-Item -Path $distfiles -ItemType Directory -Force + + foreach ($entry in $yaml['source']['files']) { + $url = $entry['url'] + $file = Split-Path $url -Leaf + $out_path = "$distfiles\$file" + $hash_file = $entry['sha256'] + $expected = (Get-Content "$dir\$hash_file").Trim() + + if (! (Test-Path $out_path)) { + Invoke-WebRequest $url -OutFile $out_path + } + + $hash = (Get-FileHash $out_path -Algorithm SHA256).Hash + if ($hash -ne $expected) { + Remove-Item $out_path + Write-Error "Invalid sha256 for downloaded '$file', aborting: got $hash, expected $expected" + } + } +} + +# TODO: make this more generic +$root = $yaml['vm']['windows'] + +. "$script_dir\common.ps1" + +# need our own EWDK environment for signing +$env:EWDK_PATH = Find-EWDK +Launch-EWDK + +# generate testsign cert +& "$script_dir\local\create-cert.ps1" "$dir\sign.crt" + +foreach ($target in $root['build']) { + # build + $args = @( + "$script_dir\build-sln.ps1", + "-solution", "$dir\$target", + "-configuration", $cfg, + "-repo", $repo, + "-testsign", + "-noisy" + ) + + if ($distfiles -ne $null) { + $args += @("-distfiles", $distfiles) + } + + $proc = Start-Process -NoNewWindow -PassThru powershell -ArgumentList $args + $proc.WaitForExit() + + # copy artifacts to local repo + $kinds = @('bin', 'inc', 'lib') + foreach ($kind in $kinds) { + New-Item -Path "$repo_dir\$kind" -ItemType Directory -Force + foreach ($output in $root[$kind]) { + # TODO: make this more generic + $output = ReplacePlaceholders $output + + if ($kind -eq "bin") { + # sign if needed + $do_sign = $false + @(".exe", ".dll", ".sys", ".cat") | % { $do_sign = $do_sign -or $output.EndsWith($_) } + + if ($do_sign) { + foreach ($skip in $root['skip-test-sign']) { + $skip = ReplacePlaceholders $skip + if ($output -eq $skip) { + $do_sign = $false + break + } + } + + if ($do_sign) { + & "$script_dir\local\sign.ps1" "$dir\sign.crt" "$dir\$output" + } + } + } + + Copy-Item "$dir\$output" "$repo_dir\$kind" + } + } + + # copy testsign cert to local repo + Copy-Item "$dir\sign.crt" $repo_dir + # delete testsign cert from OS store + & "$script_dir\local\delete-cert.ps1" "$dir\sign.crt" +} diff --git a/qubesbuilder/plugins/build_windows/scripts/local/create-cert.ps1 b/qubesbuilder/plugins/build_windows/scripts/local/create-cert.ps1 new file mode 100755 index 00000000..70ee34d3 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/local/create-cert.ps1 @@ -0,0 +1,12 @@ +# create test-sign certificate +# the private part is saved in the certificate store +# usage: $0 + +$cert_path = $args[0] + +$cn = "Qubes Tools" +$end_date = (Get-Date).AddYears(5) + +$cert = New-SelfSignedCertificate -KeyUsage DigitalSignature -KeySpec Signature -Type CodeSigningCert -HashAlgorithm sha256 -CertStoreLocation "Cert:\CurrentUser\My" -Subject $cn -NotAfter $end_date + +Export-Certificate -Cert $cert -FilePath $cert_path diff --git a/qubesbuilder/plugins/build_windows/scripts/local/delete-cert.ps1 b/qubesbuilder/plugins/build_windows/scripts/local/delete-cert.ps1 new file mode 100755 index 00000000..13d51e27 --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/local/delete-cert.ps1 @@ -0,0 +1,24 @@ +# Deletes a public certificate file and its private key from the certificate store. +# Usage: $0 [public cert] + +$ErrorActionPreference = 'Stop' + +$cert_path = "qwt.cer" +if ($args[0] -ne $null) { + $cert_path = $args[0] +} + +if (! (Test-Path $cert_path)) { + Write-Host "$cert_path not found, doing nothing" + exit 0 +} + +$tp = (Get-PfxCertificate -FilePath $cert_path).Thumbprint + +Remove-Item $cert_path + +# remove from personal cert store +Remove-Item "cert:\CurrentUser\My\$tp" + +# signtool adds it to the user's CA store so remove from there as well +Remove-Item "cert:\CurrentUser\CA\$tp" diff --git a/qubesbuilder/plugins/build_windows/scripts/local/sign.ps1 b/qubesbuilder/plugins/build_windows/scripts/local/sign.ps1 new file mode 100755 index 00000000..7bb446bd --- /dev/null +++ b/qubesbuilder/plugins/build_windows/scripts/local/sign.ps1 @@ -0,0 +1,33 @@ +# Sign the target. +# Usage: $0 +# The corresponding private key must reside in the OS certificate store. +# THIS SCRIPT IS MEANT TO BE USED ONLY FOR LOCAL TEST BUILDS. + +$ErrorActionPreference = 'Stop' + +$ts_url = "http://timestamp.digicert.com" + +if ($env:EWDK_PATH -eq $null) { + Write-Error "EWDK_PATH variable not set" +} + +if ($env:Version_Number -eq $null) { + Write-Error "EWDK environment not initialized" +} + +$signtool = "$env:EWDK_PATH\Program Files\Windows Kits\10\bin\$env:Version_Number\x64\signtool.exe" +if (! (Test-Path $signtool)) { + Write-Error "$signtool not found" + break +} + +$cert_path = $args[0] +if (! (Test-Path $cert_path)) { + Write-Error "$cert_path not found" + exit 1 +} +$sha1 = (Get-FileHash $cert_path -Algorithm SHA1).Hash + +$target = $args[1] + +Start-Process -FilePath $signtool -Wait -NoNewWindow -ArgumentList "sign /sha1 $sha1 /fd sha256 /td sha256 /tr $ts_url $target" diff --git a/qubesbuilder/plugins/source_windows/__init__.py b/qubesbuilder/plugins/source_windows/__init__.py new file mode 100644 index 00000000..658924d5 --- /dev/null +++ b/qubesbuilder/plugins/source_windows/__init__.py @@ -0,0 +1,117 @@ +# The Qubes OS Project, http://www.qubes-os.org +# +# Copyright (C) 2022 Frédéric Pierret (fepitre) +# Copyright (C) 2024 Rafał Wojdyła +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program. If not, see . +# +# SPDX-License-Identifier: GPL-3.0-or-later + +import shutil +import tempfile +from pathlib import Path + +from qubesbuilder.common import is_filename_valid +from qubesbuilder.component import QubesComponent +from qubesbuilder.config import Config +from qubesbuilder.distribution import QubesDistribution +from qubesbuilder.plugins import WindowsDistributionPlugin, PluginDependency +from qubesbuilder.plugins.source import SourcePlugin, SourceError + + +class WindowsSourcePlugin(WindowsDistributionPlugin, SourcePlugin): + """ + Manage Windows distribution source. + """ + + name = "source_windows" + stages = ["prep"] + dependencies = [PluginDependency("fetch"), PluginDependency("source")] + + def __init__( + self, + component: QubesComponent, + dist: QubesDistribution, + config: Config, + stage: str, + **kwargs, + ): + super().__init__( + component=component, dist=dist, config=config, stage=stage + ) + + def run(self): + """ + Run plugin for given stage. + """ + # Run stage defined by parent class + super().run() + + if self.stage != "prep" or not self.has_component_packages("prep"): + return + + parameters = self.get_parameters(self.stage) + + # Check if we have distribution related content defined + if not parameters.get("build", []): + self.log.info(f"{self.component}:{self.dist}: Nothing to be done.") + return + + # Compare previous artifacts hash with current source hash + hash = self.get_dist_artifacts_info( + self.stage, self.component.name + ).get("source-hash", None) + if self.component.get_source_hash() == hash: + self.log.info( + f"{self.component}:{self.dist}: Source hash is the same as already prepared source. Skipping." + ) + return + + # Get fetch info + fetch_info = self.get_dist_artifacts_info( + "fetch", + "source", + artifacts_dir=self.get_component_artifacts_dir("fetch"), + ) + + artifacts_dir = self.get_dist_component_artifacts_dir(self.stage) + + # Clean previous build artifacts + if artifacts_dir.exists(): + shutil.rmtree(artifacts_dir.as_posix()) + artifacts_dir.mkdir(parents=True) + + # Save package information we parsed for next stages + info = fetch_info + info.update( + { + "source-hash": self.component.get_source_hash(), + } + ) + + self.save_dist_artifacts_info( + stage=self.stage, basename=self.component.name, info=info + ) + + # TODO: create source archive if needed + + # save dummy per-target info, unused but base build plugin requires it + for target in parameters["build"]: + # this dummy info can't be an empty dict + self.save_dist_artifacts_info( + stage=self.stage, basename=target.mangle(), info={"dummy": 1} + ) + + +PLUGINS = [WindowsSourcePlugin] diff --git a/rpc/policy/50-qubesbuilder.policy b/rpc/policy/50-qubesbuilder.policy index 345ac5cf..ca781415 100644 --- a/rpc/policy/50-qubesbuilder.policy +++ b/rpc/policy/50-qubesbuilder.policy @@ -1,9 +1,11 @@ admin.vm.CreateDisposable * work-qubesos dom0 allow target=dom0 admin.vm.CreateDisposable * work-qubesos qubes-builder-dvm allow target=dom0 +admin.vm.CurrentState * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 admin.vm.List * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 admin.vm.Start * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 admin.vm.Kill * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 +admin.vm.Remove * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 qubesbuilder.FileCopyIn * work-qubesos @tag:disp-created-by-work-qubesos allow qubesbuilder.FileCopyOut * work-qubesos @tag:disp-created-by-work-qubesos allow diff --git a/rpc/policy/51-qubesbuilder-windows.policy b/rpc/policy/51-qubesbuilder-windows.policy new file mode 100644 index 00000000..6399b267 --- /dev/null +++ b/rpc/policy/51-qubesbuilder-windows.policy @@ -0,0 +1,15 @@ +admin.vm.device.block.Attach * work-qubesos @tag:disp-created-by-work-qubesos allow target=dom0 +qubesbuilder.WinSign.Timestamp * work-qubesos @tag:disp-created-by-work-qubesos allow +qubesbuilder.WinFileCopyIn * work-qubesos @tag:disp-created-by-work-qubesos allow +qubesbuilder.WinFileCopyOut * work-qubesos @tag:disp-created-by-work-qubesos allow + +admin.vm.device.block.Available * work-qubesos work-qubesos allow target=dom0 + +admin.vm.Start * work-qubesos win-build allow target=dom0 +admin.vm.device.block.Attach * work-qubesos win-build allow target=dom0 + +qubesbuilder.WinSign.QueryKey +Qubes__Windows__Tools work-qubesos vault-sign allow +qubesbuilder.WinSign.CreateKey +Qubes__Windows__Tools work-qubesos vault-sign allow +qubesbuilder.WinSign.DeleteKey +Qubes__Windows__Tools work-qubesos vault-sign allow +qubesbuilder.WinSign.GetCert +Qubes__Windows__Tools work-qubesos vault-sign allow +qubesbuilder.WinSign.Sign +Qubes__Windows__Tools work-qubesos vault-sign allow diff --git a/rpc/qubesbuilder-file-copy-in.ps1 b/rpc/qubesbuilder-file-copy-in.ps1 new file mode 100644 index 00000000..5abfd67a --- /dev/null +++ b/rpc/qubesbuilder-file-copy-in.ps1 @@ -0,0 +1,22 @@ +. $env:QUBES_TOOLS\qubes-rpc-services\VMExec-Decode.ps1 +. $env:QUBES_TOOLS\qubes-rpc-services\log.ps1 + +LogStart + +try { + $decoded = VMExec-Decode $args[0] + LogDebug "decoded: $decoded" + + $fileReceiver = Join-Path $env:QUBES_TOOLS "qubes-rpc-services\file-receiver.exe" + + # Create destination directory + New-Item -ItemType Directory -Path $decoded -Force | Out-Null + + $parent = Split-Path -Parent $decoded + LogDebug "parent: $parent" + # All Windows RPC executables use | as argument separator and powershell adds an extra space to the command line + # see https://github.com/PowerShell/PowerShell/issues/13094 + Start-Process -FilePath $fileReceiver -ArgumentList "$parent|" -LoadUserProfile -NoNewWindow -Wait +} catch [DecodeError] { + Write-Error $_.Exception.Message +} diff --git a/rpc/qubesbuilder-file-copy-out.ps1 b/rpc/qubesbuilder-file-copy-out.ps1 new file mode 100644 index 00000000..644a63b1 --- /dev/null +++ b/rpc/qubesbuilder-file-copy-out.ps1 @@ -0,0 +1,14 @@ +. $env:QUBES_TOOLS\qubes-rpc-services\VMExec-Decode.ps1 +. $env:QUBES_TOOLS\qubes-rpc-services\log.ps1 + +LogStart + +try { + $decoded = VMExec-Decode $args[0] + LogDebug "decoded: $decoded" + + $fileSender = Join-Path $env:QUBES_TOOLS "qubes-rpc-services\file-sender.exe" + Start-Process -FilePath $fileSender -ArgumentList "$decoded" -LoadUserProfile -NoNewWindow -Wait +} catch [DecodeError] { + Write-Error $_.Exception.Message +} diff --git a/rpc/qubesbuilder.WinFileCopyIn b/rpc/qubesbuilder.WinFileCopyIn new file mode 100644 index 00000000..a9d0620d --- /dev/null +++ b/rpc/qubesbuilder.WinFileCopyIn @@ -0,0 +1 @@ +c:\windows\system32\cmd.exe /c powershell.exe -executionpolicy bypass -noninteractive -file "%QUBES_TOOLS%\qubes-rpc-services\qubesbuilder-file-copy-in.ps1" "%1" \ No newline at end of file diff --git a/rpc/qubesbuilder.WinFileCopyOut b/rpc/qubesbuilder.WinFileCopyOut new file mode 100644 index 00000000..53933d59 --- /dev/null +++ b/rpc/qubesbuilder.WinFileCopyOut @@ -0,0 +1 @@ +c:\windows\system32\cmd.exe /c powershell.exe -executionpolicy bypass -noninteractive -file "%QUBES_TOOLS%\qubes-rpc-services\qubesbuilder-file-copy-out.ps1" "%1" \ No newline at end of file diff --git a/rpc/qubesbuilder.WinSign.CreateKey b/rpc/qubesbuilder.WinSign.CreateKey new file mode 100755 index 00000000..36fd4ed3 --- /dev/null +++ b/rpc/qubesbuilder.WinSign.CreateKey @@ -0,0 +1,47 @@ +#!/bin/bash + +set -efo pipefail + +# shellcheck source=SCRIPTDIR/qubesbuilder.WinSign.common +. "$(dirname "$0")/qubesbuilder.WinSign.common" + +usage() { + >&2 echo "Usage: $(basename "$0")+ + +Create a new self-signed key and certificate. +" + exit 1 +} + +if [ $# -ne 1 ]; then + usage +fi + +ensure_db + +KEY_NAME="${1//__/ }" + +set +e + +if check_key_exists "${KEY_NAME}"; then + >&2 echo "Key '$1' exists" + exit 1 +fi +set -e + +# Generate self-signed certificate and key +openssl req \ + -nodes \ + -new \ + -x509 \ + -newkey rsa:4096 \ + -sha256 \ + -keyout "${KEYS_DIR}/${KEY_NAME}.key" \ + -out "${KEYS_DIR}/${KEY_NAME}.crt" \ + -days "${DAYS}" \ + -subj "/CN=${KEY_NAME}/" \ + -addext "basicConstraints=CA:FALSE" \ + > /dev/null 2>&1 + +# Verify that the key got added +check_key_exists "${KEY_NAME}" diff --git a/rpc/qubesbuilder.WinSign.DeleteKey b/rpc/qubesbuilder.WinSign.DeleteKey new file mode 100755 index 00000000..d695a4dd --- /dev/null +++ b/rpc/qubesbuilder.WinSign.DeleteKey @@ -0,0 +1,33 @@ +#!/bin/bash + +set -efo pipefail + +# shellcheck source=SCRIPTDIR/qubesbuilder.WinSign.common +. "$(dirname "$0")/qubesbuilder.WinSign.common" + +usage() { + >&2 echo "Usage: $(basename "$0")+ + +Delete a signing key and certificate. +" + exit 1 +} + +if [ $# -ne 1 ]; then + usage +fi + +ensure_db + +KEY_NAME="${1//__/ }" + +set +e +if ! check_key_exists "${KEY_NAME}"; then + >&2 echo "Key '$1' does not exist" + exit 1 +fi +set -e + +sudo rm -f \ + "${KEYS_DIR}/${KEY_NAME}.key" \ + "${KEYS_DIR}/${KEY_NAME}.crt" diff --git a/rpc/qubesbuilder.WinSign.GetCert b/rpc/qubesbuilder.WinSign.GetCert new file mode 100755 index 00000000..66433f6e --- /dev/null +++ b/rpc/qubesbuilder.WinSign.GetCert @@ -0,0 +1,31 @@ +#!/bin/bash + +set -efo pipefail + +# shellcheck source=SCRIPTDIR/qubesbuilder.WinSign.common +. "$(dirname "$0")/qubesbuilder.WinSign.common" + +usage() { + >&2 echo "Usage: $(basename "$0")+ + +Get public certificate for a given signing key. +" + exit 1 +} + +if [ $# -ne 1 ]; then + usage +fi + +ensure_db + +KEY_NAME="${1//__/ }" + +set +e +if ! check_key_exists "${KEY_NAME}"; then + >&2 echo "Key '$1' does not exist" + exit 1 +fi +set -e + +cat "${KEYS_DIR}/${KEY_NAME}.crt" diff --git a/rpc/qubesbuilder.WinSign.QueryKey b/rpc/qubesbuilder.WinSign.QueryKey new file mode 100755 index 00000000..4b092dda --- /dev/null +++ b/rpc/qubesbuilder.WinSign.QueryKey @@ -0,0 +1,27 @@ +#!/bin/bash + +set -efo pipefail + +# shellcheck source=SCRIPTDIR/qubesbuilder.WinSign.common +. "$(dirname "$0")/qubesbuilder.WinSign.common" + +usage() { + >&2 echo "Usage: $(basename "$0")+ + +Query whether the given signing key exists. +" + exit 1 +} + +if [ $# -ne 1 ]; then + usage +fi + +ensure_db + +set +e +if check_key_exists "${1//__/ }"; then + echo "Key '$1' exists" +else + echo "Key '$1' does not exist" +fi diff --git a/rpc/qubesbuilder.WinSign.Sign b/rpc/qubesbuilder.WinSign.Sign new file mode 100755 index 00000000..bc35fdee --- /dev/null +++ b/rpc/qubesbuilder.WinSign.Sign @@ -0,0 +1,62 @@ +#!/bin/bash + +set -efo pipefail + +# shellcheck source=SCRIPTDIR/qubesbuilder.WinSign.common +. "$(dirname "$0")/qubesbuilder.WinSign.common" + +usage() { + >&2 echo "Usage: $(basename "$0")+ + +Sign a binary with the given key. Input = stdin, output = stdout. +" + exit 1 +} + +if [ $# -ne 1 ]; then + usage +fi + +ensure_db + +KEY_NAME="${1//__/ }" + +set +e +if ! check_key_exists "${1//__/ }"; then + >&2 echo "Key '$1' does not exist" + exit 1 +fi +set -e + +PAYLOAD_DIR="$(mktemp -d)" + +cleanup() { + local payload_dir="$1" + if [ -n "${payload_dir}" ]; then + rm -rf "${payload_dir}" + fi +} + +# expanding PAYLOAD_DIR early is the expected behavior +# shellcheck disable=SC2064 +trap "cleanup ${PAYLOAD_DIR}" EXIT + +payload="${PAYLOAD_DIR}/payload" + +# Limit stdin size +head --bytes=100MB > "$payload" + +if [ "$(stat --format=%s "$payload")" -ge $((100 * 1024 * 1024)) ]; then + >&2 echo "Input size must be less than 100MiB." + exit 1 +fi + +osslsigncode sign \ + -certs "${KEYS_DIR}/${KEY_NAME}.crt" \ + -key "${KEYS_DIR}/${KEY_NAME}.key" \ + -h sha256 \ + -in "$payload" \ + -out "$payload".signed \ + > /dev/null 2>&1 + +cat "$payload".signed diff --git a/rpc/qubesbuilder.WinSign.Timestamp b/rpc/qubesbuilder.WinSign.Timestamp new file mode 100755 index 00000000..50aa1941 --- /dev/null +++ b/rpc/qubesbuilder.WinSign.Timestamp @@ -0,0 +1,35 @@ +#!/bin/bash + +set -efo pipefail + +TS_URL="http://timestamp.digicert.com" +PAYLOAD_DIR="$(mktemp -d)" + +cleanup() { + local payload_dir="$1" + if [ -n "${payload_dir}" ]; then + rm -rf "${payload_dir}" + fi +} + +# expanding PAYLOAD_DIR early is the expected behavior +# shellcheck disable=SC2064 +trap "cleanup ${PAYLOAD_DIR}" EXIT + +payload="${PAYLOAD_DIR}/payload" + +# Limit stdin size +head --bytes=100MB > "$payload" + +if [ "$(stat --format=%s "$payload")" -ge $((100 * 1024 * 1024)) ]; then + >&2 echo "Input size must be less than 100MiB." + exit 1 +fi + +osslsigncode add \ + -ts "${TS_URL}" \ + -in "$payload" \ + -out "$payload".signed \ + > /dev/null 2>&1 + +cat "$payload".signed diff --git a/rpc/qubesbuilder.WinSign.common b/rpc/qubesbuilder.WinSign.common new file mode 100644 index 00000000..dd1f11bb --- /dev/null +++ b/rpc/qubesbuilder.WinSign.common @@ -0,0 +1,16 @@ +#!/bin/bash + +set -efo pipefail + +KEYS_DIR=/home/user/win-sign/keys + +# Validity period for generated self-signed certificates +export DAYS=3650 + +ensure_db() { + mkdir -p "${KEYS_DIR}" +} + +check_key_exists() { + [ -f "${KEYS_DIR}/$1".key ] +} diff --git a/tests/test_executors.py b/tests/test_executors.py index 4297a3dd..32116491 100644 --- a/tests/test_executors.py +++ b/tests/test_executors.py @@ -379,4 +379,4 @@ def test_qubes_on_error_noclean(): check=True, ) - executor.cleanup(dispvm) + executor.cleanup() diff --git a/tools/windows/.gitignore b/tools/windows/.gitignore new file mode 100644 index 00000000..c3e3e8ea --- /dev/null +++ b/tools/windows/.gitignore @@ -0,0 +1,3 @@ +/ewdk.iso +/win-build.iso +/win-opensshd.msi diff --git a/tools/windows/deps.txt b/tools/windows/deps.txt new file mode 100644 index 00000000..0df3e7e1 --- /dev/null +++ b/tools/windows/deps.txt @@ -0,0 +1,2 @@ +6333f5b38313360a2c960ea7afa3086002f479bad663f20f92a4889063fe925c win-opensshd.msi https://github.com/PowerShell/Win32-OpenSSH/releases/download/v9.5.0.0p1-Beta/OpenSSH-Win64-v9.5.0.0.msi +9a23f3399bf6b80b119bcaf9def8679ed296f4b7d742e0251cf0b76a3fb97f77 ewdk.iso https://go.microsoft.com/fwlink/?linkid=2271957 diff --git a/tools/windows/dom0/create-vm.sh b/tools/windows/dom0/create-vm.sh new file mode 100755 index 00000000..48257309 --- /dev/null +++ b/tools/windows/dom0/create-vm.sh @@ -0,0 +1,120 @@ +#!/bin/bash + +set -ef + +VM_NAME="win-build" +VM_LABEL="purple" +VM_MEMORY=4096 +VM_CPUS=2 +VM_SIZE="40GiB" +BUILD_VM_NAME="work-qubesos" +VM_USER="user" + +# make sure this matches the path in create-build-vm.sh +BUILD_SSH_KEY="/home/user/.ssh/win-build.key" + +usage() { + echo "Usage: $(basename "$0") [OPTIONS] ... + +This script creates a Windows builder qube. + +Options: + --iso Path to Windows image prepared for unattended setup (required) + --name Qube name (default: '$VM_NAME') + --label Qube label (default: '$VM_LABEL') + --memory RAM amount (MiB) (default: $VM_MEMORY) + --cpus Number of vcpus (default: $VM_CPUS) + --build-vm-name Name of the main builder qube (default: '$BUILD_VM_NAME') +" +} + +if ! OPTS=$(getopt -o hi:n:l:m:c:b: --long help,iso:,name:,label:,memory:,cpus:,build-vm-name: -n "$0" -- "$@"); then + exit 1 +fi + +eval set -- "$OPTS" + +while [[ $# -gt 0 ]]; do + case "$1" in + -h | --help) usage; exit 0 ;; + -i | --iso) ISO="$2"; shift ;; + -n | --name) VM_NAME="$2"; shift ;; + -l | --label) VM_LABEL="$2"; shift ;; + -m | --memory) VM_MEMORY="$2"; shift ;; + -c | --cpus) VM_CPUS="$2"; shift ;; + -b | --build-vm-name) BUILD_VM_NAME="$2"; shift ;; + esac + shift +done + +if [ -z "$ISO" ]; then + usage && exit 1 +fi + +set +e + +if qvm-check "$VM_NAME" 2> /dev/null; then + echo "[!] Qube $VM_NAME already exists!" + exit 1 +fi +set -e + +BUILD_VM_IP=$(qvm-prefs "$BUILD_VM_NAME" ip) +FW_VM_NAME=$(qvm-prefs "$BUILD_VM_NAME" netvm) + +echo "[*] Creating qube: $VM_NAME" +qvm-create --class StandaloneVM \ + --property memory="$VM_MEMORY" \ + --property vcpus="$VM_CPUS" \ + --property stubdom_mem=1024 \ + --property virt_mode=hvm \ + --property kernel='' \ + --property netvm="$FW_VM_NAME" \ + --label "$VM_LABEL" \ + "$VM_NAME" + +qvm-volume extend "$VM_NAME:root" "$VM_SIZE" + +echo "[*] Configuring firewall" +# disallow outbound network connections from the windows vm +qvm-firewall "$VM_NAME" del --rule-no 0 # default "allow" rule +qvm-firewall "$VM_NAME" add drop + +# allow network connections from the main builder vm to the windows vm +VM_IP=$(qvm-prefs "$VM_NAME" ip) + +# TODO: use a custom fw chain +set +e + +if ! qvm-run -p "$FW_VM_NAME" "sudo nft list chain ip qubes custom-forward" | grep "ip saddr $BUILD_VM_IP ip daddr $VM_IP .* accept"; then + qvm-run -p "$FW_VM_NAME" "sudo nft add rule ip qubes custom-forward ip saddr $BUILD_VM_IP ip daddr $VM_IP ct state new,established,related counter accept" +fi +set -e + +ssh_check() { + qvm-run -p "$BUILD_VM_NAME" \ + "ssh -q -o 'BatchMode yes' -o 'StrictHostKeyChecking accept-new' -o 'ConnectTimeout 10' -i $BUILD_SSH_KEY ${VM_USER}@${VM_IP} exit" +} + +# prep the main builder vm for ssh connection +qvm-run -p "$BUILD_VM_NAME" "ssh-keygen -R $VM_IP" + +# unattended Windows install +echo "[*] Installing Windows, this will take a while..." +qvm-start --cdrom="$ISO" "$VM_NAME" +FINISHED=0 +set +e +while [ "$FINISHED" == "0" ]; do + if qvm-check --running "$VM_NAME" 2> /dev/null; then + if ssh_check; then + FINISHED=1 + fi + else + qvm-start "$VM_NAME" + fi +done + +# shutdown, builder will attach the EWDK iso +qvm-shutdown --wait "$VM_NAME" + +echo "[*] Windows builder qube created successfully" diff --git a/tools/windows/edit-iso-dispvm.sh b/tools/windows/edit-iso-dispvm.sh new file mode 100644 index 00000000..9df3ef6c --- /dev/null +++ b/tools/windows/edit-iso-dispvm.sh @@ -0,0 +1,43 @@ +#!/bin/bash + +# This script is run in a dispvm + +set -efo pipefail + +ISO_DEV="/dev/xvdi" +ISO_FILES="/home/user/iso" +OUTPUT="/home/user/win-build.iso" + +INPUT_DIR="$(mktemp -d -p ~)" +OUTPUT_DIR="$(mktemp -d -p ~)" + +sudo mount -r "${ISO_DEV}" "${INPUT_DIR}" + +echo "[*] Extracting unmodified iso..." +cp -rp "${INPUT_DIR}/." "${OUTPUT_DIR}" +sudo umount "${ISO_DEV}" +rmdir "${INPUT_DIR}" + +echo "[*] Adding files..." +sudo cp -r "${ISO_FILES}/." "${OUTPUT_DIR}" + +# Generate random password for the Windows user +set +e # `head` below causes SEGPIPE... +WIN_PASS=$(tr -dc 'A-Za-z0-9!"#$%&'\''()*+,-.:;<=>?@[\]^_`{|}~' &2 echo "qrexec call '$2' to '$1' failed: ${result[0]} ${result[1]}" + exit 1 + fi + echo "${result[1]}" +} + +# $1=target, $2=input +shell_call() { + echo "$2" | qrexec-client-vm "$1" qubes.VMShell +} + +SCRIPT_DIR=$(dirname "$0") +SCRIPT_DIR=$(readlink -f "${SCRIPT_DIR}") + +echo "[*] Setting up a loop device for the ISO..." +LODEV=$(sudo losetup -f) +sudo losetup "${LODEV}" "${INPUT}" +LOOP_ID="${LODEV#'/dev/'}" + +echo "[*] Preparing a DispVM..." +SELF=$(qubesdb-read /name) +DISPVM=$(qrexec_call "dom0" admin.vm.CreateDisposable) + +qrexec_call "${DISPVM}" "admin.vm.Start" +qrexec_call "${DISPVM}" "admin.vm.device.block.Attach+${SELF}+${LOOP_ID}" "read-only=true" +qvm-copy-to-vm --without-progress "${DISPVM}" "${SCRIPT_DIR}/edit-iso-dispvm.sh" +qvm-copy-to-vm --without-progress "${DISPVM}" "${FILES}" +shell_call "${DISPVM}" "mv ~/QubesIncoming/${SELF}/edit-iso-dispvm.sh ~" +shell_call "${DISPVM}" "mv ~/QubesIncoming/${SELF}/$(basename "$(realpath "${FILES}")") ~/iso" +shell_call "${DISPVM}" "chmod +x ~/edit-iso-dispvm.sh" +# shellcheck disable=SC2088 # (~ expansion) +shell_call "${DISPVM}" "~/edit-iso-dispvm.sh" + +sudo losetup -d "${LODEV}" + +echo "[*] Copying the final iso from '${DISPVM}' to '${OUTPUT}'..." + +shell_call "${DISPVM}" "cat ~/win-build.iso" > "${OUTPUT}" +qrexec_call "${DISPVM}" "admin.vm.Kill" + +echo "[*] Done!" diff --git a/tools/windows/generate-iso.sh b/tools/windows/generate-iso.sh new file mode 100755 index 00000000..fc2171fb --- /dev/null +++ b/tools/windows/generate-iso.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +set -efo pipefail + +EDITED_ISO="win-build.iso" +SCRIPT_DIR=$(dirname "$0") +SCRIPT_DIR=$(readlink -f "$SCRIPT_DIR") +SSH_KEY="/home/user/.ssh/win-build.key" + +usage() { + echo "Usage: $(basename "$0") [OPTIONS] + +This script prepares an .iso image for the Windows builder executor qube. + +Options: + --iso Path to unmodified Windows installation .iso file + --output Path to output (edited) ISO file (default: ${EDITED_ISO}) +" +} + +if ! OPTS=$(getopt -o hi:o: --long help,iso:,output: -n "$0" -- "$@"); then + exit 1 +fi + +eval set -- "$OPTS" + +while [[ $# -gt 0 ]]; do + case "$1" in + -h | --help) usage; exit 0 ;; + -i | --iso) ISO="$2"; shift ;; + -o | --output) EDITED_ISO="$2"; shift ;; + esac + shift +done + +if [ -z "${ISO}" ] || [ -z "${EDITED_ISO}" ]; then + usage + exit 1 +fi + +# download/verify prerequisites +"$SCRIPT_DIR/get-files.sh" -o "$SCRIPT_DIR" "$SCRIPT_DIR/deps.txt" + +# sshd installer +cp -f "${SCRIPT_DIR}/win-opensshd.msi" "${SCRIPT_DIR}/iso-files/sources/\$OEM\$/\$1/qubes" + +# ssh key +if [ -f "${SSH_KEY}" ]; then + echo "[*] Using existing ssh key: ${SSH_KEY}" +else + echo "[*] Creating ssh key: ${SSH_KEY}" + ssh-keygen -q -t ed25519 -N '' -f "${SSH_KEY}" +fi + +cp -f "${SSH_KEY}.pub" "${SCRIPT_DIR}/iso-files/sources/\$OEM\$/\$1/qubes" + +# prepare edited iso +"${SCRIPT_DIR}/edit-iso.sh" --input "$ISO" --output "$EDITED_ISO" --files "${SCRIPT_DIR}/iso-files" diff --git a/tools/windows/get-files.sh b/tools/windows/get-files.sh new file mode 100755 index 00000000..c5523a80 --- /dev/null +++ b/tools/windows/get-files.sh @@ -0,0 +1,79 @@ +#!/bin/bash + +set -efo pipefail + +unset OUTDIR OPTS GETOPT_COMPATIBLE FILE_LIST SHA256 FILE_NAME FILE_URL FETCH_CMD UNTRUSTED_FILE_NAME LINES ROW + +OUTDIR=$(pwd) + +usage() { + echo "Usage: $(basename "$0") [OPTIONS] file_list + +This script downloads and verifies files specified in file_list. +file_list should contain lines in the following format: + + +Options: + --output-dir Output directory +" +} + +# $1 = expected sha256, $2 = file path +verify() {( + set +e + if [[ $# -ne 2 ]]; then + echo "[!] Bad arguments to verify()" + return 1 + fi + + echo -n "[*] Verifying $2 ... " + + if echo "$1 $2" | sha256sum --check --status; then + echo "OK" + else + echo "Failed!" + rm -f "$2" + return 1 + fi +)} + + +if ! OPTS=$(getopt -o ho: --long help,output-dir: -n "$0" -- "$@"); then + exit 1 +fi + +eval set -- "$OPTS" + +while [[ $# -gt 0 ]]; do + case "$1" in + -h | --help) usage; exit 0 ;; + -o | --optput-dir) OUTDIR="$2"; shift ;; + *) FILE_LIST="$1" ;; + esac + shift +done + +if [ -z "${FILE_LIST}" ]; then + usage + exit 1 +fi + +FETCH_CMD='curl --proto =https --proto-redir =https --tlsv1.2 --http1.1 -sSfL -o' + +# we don't use "while read... < $FILE_LIST" because qvm-run messes with stdin +readarray -t LINES < "${FILE_LIST}" + +for ROW in "${LINES[@]}"; do + read -r SHA256 FILE_NAME FILE_URL <<< "${ROW}" + [[ "${SHA256}" = "#"* ]] && continue + if [ -f "${OUTDIR}/${FILE_NAME}" ]; then + echo "[*] File ${FILE_NAME} already exists" + verify "${SHA256}" "${OUTDIR}/${FILE_NAME}" || exit 1 + else + echo "[*] Downloading ${FILE_NAME}..." + UNTRUSTED_FILE_NAME="untrusted_${FILE_NAME}" + # don't save the file in /tmp because it may be too large + qvm-run-vm --dispvm "${FETCH_CMD} /home/user/${FILE_NAME} ${FILE_URL} && cat /home/user/${FILE_NAME}" > "${OUTDIR}/${UNTRUSTED_FILE_NAME}" + ( verify "${SHA256}" "${OUTDIR}/${UNTRUSTED_FILE_NAME}" && mv "${OUTDIR}/${UNTRUSTED_FILE_NAME}" "${OUTDIR}/${FILE_NAME}" ) || exit 1 + fi +done diff --git a/tools/windows/iso-files/autounattend.xml b/tools/windows/iso-files/autounattend.xml new file mode 100644 index 00000000..af66d733 --- /dev/null +++ b/tools/windows/iso-files/autounattend.xml @@ -0,0 +1,211 @@ + + + + + + en-US + + 0409:00000409 + en-US + en-US + en-US + + + + + + 0 + 2 + + + + + + VK7JG-NPHTM-C97JM-9MPGT-3V66T + + true + + + + 1 + cmd.exe /c ">>"X:\diskpart.txt" echo SELECT DISK=0" + + + 2 + cmd.exe /c ">>"X:\diskpart.txt" echo CLEAN" + + + 3 + cmd.exe /c ">>"X:\diskpart.txt" echo CREATE PARTITION PRIMARY SIZE=100" + + + 4 + cmd.exe /c ">>"X:\diskpart.txt" echo FORMAT QUICK FS=NTFS LABEL="System Reserved"" + + + 5 + cmd.exe /c ">>"X:\diskpart.txt" echo ACTIVE" + + + 6 + cmd.exe /c ">>"X:\diskpart.txt" echo CREATE PARTITION PRIMARY" + + + 7 + cmd.exe /c ">>"X:\diskpart.txt" echo FORMAT QUICK FS=NTFS LABEL="Windows"" + + + 8 + cmd.exe /c "diskpart.exe /s "X:\diskpart.txt" >>"X:\diskpart.log" || ( type "X:\diskpart.log" & echo diskpart encountered an error. & pause & exit /b 1 )" + + + 9 + reg.exe add "HKLM\SYSTEM\Setup\LabConfig" /v BypassTPMCheck /t REG_DWORD /d 1 /f + + + 10 + reg.exe add "HKLM\SYSTEM\Setup\LabConfig" /v BypassSecureBootCheck /t REG_DWORD /d 1 /f + + + 11 + reg.exe add "HKLM\SYSTEM\Setup\LabConfig" /v BypassRAMCheck /t REG_DWORD /d 1 /f + + + + + + + No + + + + + + + 1 + ReAgentc.exe /disable + + + 2 + cmd.exe /c "del /a /f "C:\Windows\System32\Recovery\Winre.wim"" + + + 3 + net.exe accounts /maxpwage:UNLIMITED + + + 4 + reg.exe add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v LongPathsEnabled /t REG_DWORD /d 1 /f + + + 5 + fsutil.exe behavior set disableLastAccess 1 + + + 6 + reg.exe add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Authentication\LogonUI\BootAnimation" /v DisableStartupSound /t REG_DWORD /d 1 /f + + + 7 + reg.exe add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\EditionOverrides" /v UserSetting_DisableStartupSound /t REG_DWORD /d 1 /f + + + 8 + reg.exe add "HKLM\SYSTEM\CurrentControlSet\Control\BitLocker" /v "PreventDeviceEncryption" /t REG_DWORD /d 1 /f + + + 9 + powercfg.exe /h off + + + 10 + reg.exe add "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System" /v EnableLUA /t REG_DWORD /d 0 /f + + + 11 + reg.exe load "HKU\DefaultUser" "C:\Users\Default\NTUSER.DAT" + + + 12 + reg.exe add "HKU\DefaultUser\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced" /v "HideFileExt" /t REG_DWORD /d 0 /f + + + 13 + reg.exe add "HKU\DefaultUser\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced" /v "Hidden" /t REG_DWORD /d 1 /f + + + 14 + reg.exe add "HKU\DefaultUser\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Advanced" /v "ShowSuperHidden" /t REG_DWORD /d 1 /f + + + 15 + reg.exe unload "HKU\DefaultUser" + + + 16 + reg.exe add "HKLM\SYSTEM\CurrentControlSet\Control\Network\NewNetworkWindowOff" /f + + + + + win-build + UTC + + + 1 + + + 0 + + + 1 + + + + + 0409:00000409 + en-US + en-US + en-US + + + + + + user + Administrators + + + true</PlainText> + </Password> + </LocalAccount> + </LocalAccounts> + </UserAccounts> + <AutoLogon> + <Username>user</Username> + <Enabled>true</Enabled> + <LogonCount>10000</LogonCount> + <Password> + <Value><![CDATA[@PASSWORD@]]></Value> + <PlainText>true</PlainText> + </Password> + </AutoLogon> + <OOBE> + <ProtectYourPC>3</ProtectYourPC> + <HideEULAPage>true</HideEULAPage> + <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE> + </OOBE> + <FirstLogonCommands> + <SynchronousCommand wcm:action="add"> + <CommandLine>powershell -executionpolicy bypass -File c:\qubes\ssh.ps1</CommandLine> + <Description>Install sshd</Description> + <Order>1</Order> + <RequiresUserInput>false</RequiresUserInput> + </SynchronousCommand> + </FirstLogonCommands> + </component> + <component name="Microsoft-Windows-WinRE-RecoveryAgent" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> + <UninstallWindowsRE>true</UninstallWindowsRE> + </component> + </settings> +</unattend> diff --git a/tools/windows/iso-files/sources/$OEM$/$1/qubes/.gitignore b/tools/windows/iso-files/sources/$OEM$/$1/qubes/.gitignore new file mode 100644 index 00000000..87d41830 --- /dev/null +++ b/tools/windows/iso-files/sources/$OEM$/$1/qubes/.gitignore @@ -0,0 +1,2 @@ +/win-build.key.pub +/win-opensshd.msi diff --git a/tools/windows/iso-files/sources/$OEM$/$1/qubes/ssh.ps1 b/tools/windows/iso-files/sources/$OEM$/$1/qubes/ssh.ps1 new file mode 100644 index 00000000..fb9da46e --- /dev/null +++ b/tools/windows/iso-files/sources/$OEM$/$1/qubes/ssh.ps1 @@ -0,0 +1,27 @@ + +$src = "c:\qubes" + +Start-Process -Wait -FilePath "msiexec.exe" -ArgumentList "/i","$src\win-opensshd.msi","/passive" + +New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22 -Program 'c:\Program Files\OpenSSH\sshd.exe' + +$akeys = "c:\ProgramData\ssh\administrators_authorized_keys" +cp "$src\win-build.key.pub" "$akeys" + +# set permissions to only allow administrators +$acl = Get-Acl $akeys +$acl.SetAccessRuleProtection($true, $false) # disable ACL inheritance and remove all ACEs +$rule = New-Object System.Security.AccessControl.FileSystemAccessRule("BUILTIN\Administrators", "FullControl", "None", "None", "Allow") +$acl.SetAccessRule($rule) +$rule = New-Object System.Security.AccessControl.FileSystemAccessRule("NT Authority\SYSTEM", "FullControl", "None", "None", "Allow") +$acl.SetAccessRule($rule) +Set-Acl $akeys $acl + +$config = "c:\ProgramData\ssh\sshd_config" +cp -Force "$src\sshd_config" "$config" +# allow reading by authenticated users +$rule = New-Object System.Security.AccessControl.FileSystemAccessRule("NT Authority\Authenticated Users", "ReadAndExecute", "None", "None", "Allow") +$acl.SetAccessRule($rule) +Set-Acl $config $acl + +Restart-Service -Name sshd diff --git a/tools/windows/iso-files/sources/$OEM$/$1/qubes/sshd_config b/tools/windows/iso-files/sources/$OEM$/$1/qubes/sshd_config new file mode 100644 index 00000000..64afadd8 --- /dev/null +++ b/tools/windows/iso-files/sources/$OEM$/$1/qubes/sshd_config @@ -0,0 +1,4 @@ +PasswordAuthentication no +Subsystem sftp sftp-server.exe +Match Group administrators + AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys