v4.5.0

The Warewulf project is pleased to announce the release of v4.5.0.

There's a lot in this release! Some of the more prominent changes are described here. For a more complete list of changes, see the changelog.

Testing, linting, and code formatting

In this release we have significantly increased our focus on testing. Code changes, especially bug fixes, are increasingly accompanied by additional tests in the automated test suite, and this test suite is automatically run by GitHub against every pull request and every release.

Use the make test target to run the test suite locally whenever you're doing development of your own.

Additional Makefile targets have also been introduced (or improved) to aid in overall maintenance of the code base:

• make vet uses standard golang heuristics to flag potential trouble
• make fmt applies a uniform source code format

Documentation

Documentation continues to be an area for growth in Warewulf; but v4.5.x brings a number of improvements, some foundational:

• v4.5.x documentation is now published separately, in addition to documentation for the main branch.
• The quickstart guides have been updated, particularly combining the guides for Enterprise Linux (e.g., Rocky Linux, CentOS, and RHEL).
• We've added a guide for using Vagrant as a development environment.

There have been a few other specific additions: for example, there is now explicit documentation for how to configure multiple networks on a cluster node, and documentation for the "hostlist" syntax used by multiple wwctl commands.

Please reach out via Slack if you'd be interested in helping to improve the project's documentation!

Preview features

This release brings three new features in a preview status: they haven't quite had enough use, testing, integration, or documentation to be ready for production, but they represent interesting new capabilities for the project and we are eager to hear about your experiences!

Initialize storage with Ignition

You can now initialize file systems, partitions, and disks on attached storage during node boot using the Ignition utility. This does not yet support provisioning the node image to local disk; but it can be used to provision storage to be mounted into the node image; for example, to initialize a swap partition, to format a scratch disk, or to prepare local storage for system logs.

More information is available in the documentation.

Use dnsmasq for dhcp and tftp

You can now use dnsmasq as a dhcp and tftp server in place of dhcpd and tftp-server. dnsmasq may become the default implementation in the future, unifying Warewulf on a single code-base for some of its down-stream services.

More information is available in the documentation.

Use GRUB as a node bootloader

You can now use GNU GRUB as a node bootloader in place of iPXE. Using GRUB introduces support for secureboot, and may also help us to overcome some limitations on node image size that we have encountered with the available releases of iPXE.

More information is available in the documentation.

wwctl commands and arguments

This release introduces multiple improvements to the wwctl utility.

New commands

• wwctl container copy duplicates a container image to a new name, simplifying the process of testing a change in a test image.
• wwctl container rename adds support for renaming a container image in-place. (Nodes and profiles that refer to the old name must be updated separately.)

New command options and flags

• wwctl container syncuser --build automatically (re)builds a container image after syncuser.
• wwctl <node|profile> list --all now only shows attributes that have a value set.
  • wwctl <node|profile> list --fullall shows all attributes, including those which do not have a set value.
• wwctl <node|profile> set --primarynet sets the primary network device for the node, to be used for Warewulf provisioning.
• wwctl <node|profile> set --ipmiescapechar changes the ipmitool escape character.
• wwctl overlay import --parents automatically creates intermediate parent directories.

Fixes and improvements

• wwctl node edit is now much more reliable, with multiple bugfixes and a general rewrite of its temporary file handling.
• wwctl <node|profile> set --mtu now correctly accepts valid MTU values.
• wwctl container import can now import a container archive using a relative path.
• wwctl container import --force no longer fails when replacing an existing container.
• wwctl <node|profile> list now supports a comma-separated list of nodes or profiles to filter the list to be displayed.
• wwctl overlay tab completion now supports completing files within an overlay.

Overlays

Much of Warewulf's functionality is actually implemented using its bundled overlays.

• Nodes that use multiple profiles now include overlays from all associated profiles, applied in the order that the profiles are listed on the node (last wins). Individual overlays can be excluded using a ~ prefix.
• Built overlay images are no longer erroneously / confusingly mixed with overlay sources by default. (For example, overlay sources may be included in /var/lib/warewulf/overlays, and overlay images will now be built in /var/lib/warewulf/provision/overlays.)
• Built overlay images are now named either __SYSTEM__ or __RUNTIME__ by default, replacing the previous format that was a comma-separated list of the contained overlays.

wwinit

• Network configuration files now correctly configure MTU.
• Access to /warewulf/config is now restricted to root only.
• Configuration of network device names is now more reliable, particularly for nodes with multiple interfaces.
• ONBOOT is now correctly configured when using ifcfg.
• Bonded network interfaces can now be configured via NetworkManager.

generic

• The primary interface hostname is now used as the canonical name in /etc/hosts.
• The Warewulf server FQDN is now used as the canonical name in /etc/hosts.

Tags

Warewulf suports arbitrary key-value tags on nodes, network devices, and ipmi interfaces. These tags can then be used by built-in and custom overlays.

• Multiple bugs in the handling of node and profile tags have been fixed, making them a much more robust and reliable component of the Warewulf feature set.
• The wwinit overlay now uses DNS network tags (including DNS1, DNS2, etc.) to configure DNS resolution in network configuration files.

Configuration

Warewulf is primarily configured using two configuration files, warewulf.conf and nodes.conf.

warewulf.conf

• ipaddr now supports CIDR format, optionally inferring netmask and network.
• paths can be used to override compiled-in paths, e.g., for overlays, containers, and provisioned images.
• tftp:ipxe now supports full paths.

All available configuration options can be seen in context using wwctl genconfig warewulfconf print.

nodes.conf

• PrimaryNetDev now defines a single network interface as primary for a node (or profile), deprecating the primary boolean on each individual network device.

Build and packaging

Finally, a number of improvements have been made to the Warewulf build system, including its Makefile and RPM specfile.

• Warewulf no longer bundles an iPXE binary within its source code or packages, preferring, in stead, the version of iPXE included with the host disribution.
  • A local, updated build of iPXE can still be built using an updated helper script at scripts/build-ipxe.sh. (Use warwulf.conf:tftp:ipxe to configure Warewulf to use a local iPXE build.)
• Warewulf can now be built on Fedora.
• New releases now include a build for Rocky Linux 9.
• New releases now include a build for OpenSUSE Leap 15.5 (replacing older versions of OpenSUSE Leap).
• A nightly release is now published on GitHub.

v4.5.0rc2

Second release candidate for upcoming release v4.5.0.

For a complete list of changes, see https://github.com/warewulf/warewulf/blob/v4.5.0rc2/CHANGELOG.md

v4.5.0rc1

Release candidate for upcoming release v4.5.0.

For a complete list of changes, see https://github.com/warewulf/warewulf/blob/v4.5.0rc1/CHANGELOG.md

Warewulf nightly release

THIS IS A NIGHTLY RELEASE

This release contains latest commits, which is a feature unstable version

Reflog:
15de12e main@{0}: branch: Created from refs/remotes/origin/main

Commits:
["commit 15de12e","Merge: ec61ffb 87da57d","Author: Christian Goll [email protected]","Date: Mon Feb 10 10:37:11 2025 +0100",""," Merge pull request #1720 from anderbubble/netdev-ipcidr-empty"," "," Return "" when NetDev.IpCIDR is empty","","commit 87da57d","Author: Jonathon Anderson [email protected]","Date: Fri Feb 7 18:05:38 2025 -0700",""," Return "" when NetDev.IpCIDR is empty"," "," Signed-off-by: Jonathon Anderson [email protected]","","commit ec61ffb","Merge: 66c4e4b 5cdaa51","Author: Christian Goll [email protected]","Date: Mon Feb 10 10:24:09 2025 +0100",""," Merge pull request #1722 from anderbubble/args-and-completions"," "," wwctl args and completions","","commit 5cdaa51","Author: Christian Goll [email protected]","Date: Mon Feb 10 10:11:55 2025 +0100",""," additional malformed tests"," "," Signed-off-by: Christian Goll [email protected]"]

Bugfix release v4.4.1

Warewulf v4.4.1 is a bugfix release in the 4.4.x branch. This release backports certain fixes from the development branch.

Fixed

• Container file gids are now updated properly during syncuser. #840
• Added a missing .ww extension to the 70-ww4-netname.rules template in the wwinit overlay. #724
• Restrict access to /warewulf/config to root only. #728

v4.4.0

Final release of 4.4.0

Changes to node and profile subcommands

Warewulf 4.4.0 unifies the underlying implementation of nodes and profiles to prevent divergence in their features and attributes going forward; but this necessitated an update to their respective command-line arguments to unify them as well. Changes to those arguments is detailed here:

wwctl node add

• --discoverable becomes --discoverable=true

wwctl node set

• --addprofile has been removed
• --assetkey becomes --asset
• --delprofile has been removed
• --discoverable becomes --discoverable=true
• --netdeltag becomes --nettagdel
• --nettag becomes --nettagadd
• --system becomes --wwinit
• --tag becomes --tagadd
• --undiscoverable becomes --discoverable=false

internal/app/wwctl/profile/set/root.go

• --assetkey becomes --asset
• --discoverable becomes --discoverable=true
• --netdeltag becomes --nettagdel
• --nettag becomes --nettagadd
• --system becomes --wwinit
• --tag becomes --tagadd
• --undiscoverable becomes --discoverable=false

All changes introduced since v4.3.0 are summarized below.

Added

• iPXE binaries included with Warewulf now support VLAN tagging. #563
• wwctl container list now shows the container creation date,
  modification date, and size. #537
• wwctl node edit supports directly editing or defining node
  configuration YAML in an editor. #540
• wwctl node export and wwctl node import support importing and
  exporting node definitions as YAML or (for import) CSV. The CSV file
  must have a header in where the first field must always be the
  nodename, and the rest of the fields are the same as the long
  commandline options. Network device must have the form
  net.$NETNAME.$NETOPTION. (e.g., net.default.ipaddr) #540
• The warewulfd.service systemd unit file now supports execreload
  and execstop. #550
• Network interfaces now accept an mtu attribute. #549
• The wwinit overlay now supports network interface configuration
  via NetworkManager for Ethernet and InfiniBand interfaces. #539
• Default node attribute values (e.g., for kernel arguments) are now
  read in from a defaults.conf configuration file. If this file
  is not present, built-in default values are used. #539
• Warewulf documentation is now managed
  alongside the Warewulf source code in a single code repository so
  that documentation may be updated alongside code changes.
• New man pages for warewulf.conf and nodes.conf #510
• An initial cut of the Warewulf API #471
• wwctl show --render shows overlay templates as they would be
  rendered on a given target node. #467
• wwctl ssh now supports Bash completion. #466
• The environment variable WW_CONTAINER_SHELL is defined in a wwctl container shell environment to indicate the container in use. #579
• Network interface configuration (ifcfg) files now include the
  interface name and type. #457
• New defaults.conf man page. #593
• A new debug overlay includes a template which demonstrates accessing
  all available variables. #599
• Distribute a README along with staticfiles. #189
• Add a -y flag to wwctl profile add. #610
• Distribute a source RPM with GitHub releases. #614
• New Docker container node image for CentOS 7. #621

Changed

• wwctl overlay edit no longer saves a new template to the overlay
  if the template is not modified from its initial state. #522
• The wwinit overlay now only sets a name for a network interface if
  that interface has a MAC address defined. #553
• wwctl container delete now also deletes the built images
  associated with that container. #214
• Unified internal code paths for wwctl profile and wwctl node
  commands, and between the on disk YAML format and the in memory
  format, enabling the command-line options to be autogenerated from
  the datastructures and ensuring that profile and node capabilities
  remain in sync. Multiple command line arguments have been updated or
  changed. #495, #637
• wwctl power commands no longer separates node output with
  additional whitespace. #514
• No longer ask for confirmation when deleting 0 nodes. #603
• Ask for confirmation during wwctl container delete. #606

Fixed

• /etc/warewulf/excludes (read from the node image) once again
  excludes files from being included in the node image. #532
• wwctl ssh always uses a node's primary interface. #544
• wwctl container show now correctly shows the kernel version. #542
• System users are no longer prevented from logging into compute
  nodes. #538
• wwctl overlay chown now correctly handles uid and gid arguments. #530
• wwctl overlay chown no longer sets gid to 0 when unspecified. #531
• Corrected the path for .wwbackup files in some situations. #524
• Bypass imgextract for legacy BIOS machines to avoid 32-bit memory
  limitations. #497
• warewulfd no longer panics when network interface tags are
  defined. #468
• The wwinit overlay now configures the network device type. #465
• Minor typographical fixes. #528, #519
• Work-around for older versions of gzip that lack a --keep flag
  during wwctl container build. #580
• The default ipxe template is once again specified as a built-in
  default and in defaults.conf. #581
• wwctl container list no longer segfaults when a container chroot
  is present without a built image. #585
• wwctl configure hostfile now correctly detects the presence of the
  hostfile overlay template. #571
• wwctl overlay build no longer panics when rendering an template
  for a node which has tags set. #568
• Minor typographical fixes. #569
• Directories within overlays no longer lose group/other write permissions #584
• wwctl profile set now indicates "profiles" in output where it
  previously mistakenly indicated "nodes." #600
• Set correct overlay permissions for a NetworkManager configuration
  file. #591
• Replaced an invalid variable name in a NetworkManager overlay
  template. #626
• The 'nodes' alias now correctly refers to 'node' rather than
  'profile'.
• Fixed a typo in a log message. #631
• Boolean attributes now correctly account for profile and default
  values. #630
• Kernel version is shown correctly for symlink'd kernels #640

v4.4.0rc3

Hopefully the last prerelease if no significant bugs are encountered.

Second release candidate of 4.4.0

Fixes

• The ipxe template did not have a default value so nodes did not boot if a this value was not set. Please check your defaults.conf if you installed this release
• older gzip versions are now handled correctly
• set WW_CONTAINER_SHELL to container name on wwctl container exe

4.4.0rc1

First release candidate of the 4.3.0 branch.

Changes

New features

• Changed internal interacation between the on disk YAML format and the in memory
  format. This enables the command line options to be autogenerated from the datastructures.
  Also this keeps the options for profiles and node set/add commands in strict sync.
• Added node edit/export/import where the edit command exposes the YAML config for the
  given nodes in an editor. Nodes can also be added with this command. The export command
  simply drops the YAML config for the given nodes. With the import command nodes can be
  imported as YAML or as a CSV file. The CSV file must havr a header in where the first field
  must always be the nodename, the rest of the fields are the same as the long commandline
  options. Network device must have the form net.$NETNAME.$NETOPTION, which would e.g. be
  net.default.ipaddr
• The default value (like kernel args) are no read in from the configuration file
  default.conf. If this file couldn't be read built in default values will be used.
• Template files for NetworkManager are now present in the wwinit overlay. A interface must
  have the type ethernet or infiniband that these will be brought up by NetworkManager
• Listings of container via wwctl container list will now show the creation,modification
  date and the size.
• Initial cut of the warewulf API. See #471

Bug fixes

• For the connections of wwctl ssh the primary interface will be used
• Fixed kernel version in wwctl container show
• remove the container image and not only the rootfs dir on delete
• include and exclude handling for container were fixed

Warewulf 4.3.0

The Warewulf development team is happy to release Warewulf version 4.3.0!

Changes

• The kernel is now part of the container by default which means the entire kernel and user space stack can be built, maintained, and distributed within a single container. It is still possible to override a specific kernel using the --kerneloverride option to nodes and profiles.
• Network interfaces are no longer named as the "device name", but rather have a free from string as the network name with the initial name being "default".
• Creating of '/etc/exports' can now be disabled, so that wwctl configure -a wont overwrite a existing '/etc/exports'.
• All configurations files for the host (/etc/exports, /etc/dhcpd.conf, /etc/hosts) are now populated from the host overlay. Also other configuration files like prometheus.yml.ww or slurm.conf.ww which depend on the cluster nodes can be placed there.
• New template functions now exist: {{ abort }}, {{ IncludeBlock }} and {{ no_backup }} are now supported.
• Nodes can now have multiple system and runtime overlays, as a comma separated list.
• Simple ipv6 support is now enabled. In warewulf.conf the option ipaddr6/Ipv6net must be set to enable ipv6.
• Multiple files can now be created from a single template overlay file with the {{ file FILENAME }} command in the template. The command is expanded to the magic template command {{ /* file FILENAME */}} which is picked up by wwctl and everything which comes after this magic comment will be written to the file FILENAME. This mechanism is leveraged in the configuration files for the network, see ifcfg.xml.ww and ifcgf.ww.
• Networks can now have arbitrary keys value pairs in the profiles and on the node, so that things like bridges and mtu sizes can be set
• The yaml format for nodes has now sub-keys for ipmi and kernel, old nodes.conf files have to be changed accordingly
• uids and gids of a container now get synced at import time, so that at least users with the same name have the same uid. This is not necessarily needed for warewulf, but services like munge.
• Refactoring of the Warewulf daemon to support better concurrency and building overlays on the fly.

If you have any problems please join our Slack channel and/or submit an issue here: https://github.com/hpcng/warewulf/issues

This is also release has also breaking changes for the configuration files, as the format for the network devices in the nodes.conf has changed, Also warewulf.conf has a new way to configure the nfs exports on the master node and separate sections for the ipmi and kernel configs.

note: From this release on, the main branch will only get bug fixes until the next release. Development will happen in the develop branch.