Consider disabling SSH password login by default

[bgilbert]

Ignition configs allow setting user passwords. There are (at least) two orthogonal reasons to do this:

1. Being able to log in on a serial or VGA console.
2. Being able to log in via SSH password auth.

The first case is useful as a debugging mechanism. The second seems like a model we'd want to discourage. Currently, by default, enabling the first case implies enabling the second case.

Consider disabling SSH password auth by default:

PasswordAuthentication no
# Disable password login as root by default, even if passwords are re-enabled generally
PermitRootLogin prohibit-password

It should be possible (and documented) to override these settings via an Ignition config. However, sshd_config currently doesn't support inclusion of config fragments. Thus we appear to have several suboptimal choices:

1. Document re-enabling password auth by modifying the monolithic sshd_config with Ignition.

   1. Document replacing the entire file.
   2. Remove uncommented PasswordAuthentication and PermitRootLogin directives from sshd_config (to prevent them from taking precedence) and document appending to the file.

   This will prevent future sshd_config updates from taking effect on affected machines.

2. Carry a downstream patch to OpenSSH.

3. Build a bespoke mechanism for configuring password authentication, using e.g. a systemd EnvironmentFile and -o options to sshd. Automatically migrate this configuration to config fragments when that functionality becomes available.

4. Leave SSH password auth enabled in the initial FCOS release. This probably precludes disabling it in the future.

[dustymabe]

+1 for PasswordAuthentication no and PermitRootLogin prohibit-password.

I added a comment to the sshd bugzilla to express our interest in the feature and to see how likely or unlikely that RFE is.

[bgilbert]

Pending support in upstream sshd, we could emulate support for config fragments. Quick hack:

# /etc/systemd/system/sshd-make-config.service

[Unit]
Description=OpenSSH server daemon config generator
Before=sshd.service
PartOf=sshd.service
ReloadPropagatedFrom=sshd.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/mkdir -m 700 -p /run/sshd
ExecStart=/bin/touch /run/sshd/sshd_config
ExecStart=/bin/chmod 600 /run/sshd/sshd_config
ExecStart=/bin/bash -c 'shopt -s nullglob && cat /etc/ssh/sshd_config.d/*.conf /etc/ssh/sshd_config > /run/sshd/sshd_config'
ExecReload=/bin/bash -c 'shopt -s nullglob && cat /etc/ssh/sshd_config.d/*.conf /etc/ssh/sshd_config > /run/sshd/sshd_config'

# /etc/systemd/system/sshd.service.d/conf.conf

[Unit]
Requires=sshd-make-config.service

[Service]
ExecStart=
ExecStart=/usr/sbin/sshd -D $OPTIONS $CRYPTO_POLICY -f /run/sshd/sshd_config

That appears to correctly update sshd_config prior to start, restart, and reload of sshd.service.

[bgilbert]

Discussed in today's meeting.

It turns out that the upstream defaults are:

PasswordAuthentication yes
PermitRootLogin prohibit-password

but Fedora changes the latter to PermitRootLogin yes.

There was previously a Fedora Change proposal to disable SSH root login (prior discussion, Change discussion, related bug). Discussion of that Change centered on the security value of disabling passwords only for root, the ability to SSH into the machine if no user account was created, and the ability to enroll an SSH key without a way to get into the machine. None of those issues apply here.

No one in today's meeting indicated a preference for enabling password auth by default. Next steps:

1. Decide how to change the config in a way that can be cleanly overridden with Ignition
2. Make the change and document it

I'll defer PRing the design doc for a bit to allow for discussion of implementation approaches.

[dustymabe]

but Fedora changes the latter to PermitRootLogin yes.

FYI: Fedora 31 change proposal to disable root password via SSH: https://fedoraproject.org/wiki/Changes/DisableRootPasswordLoginInSshd

[bgilbert]

PR in coreos/fedora-coreos-config#96 to disable password auth. Without sshd_config fragment support, overriding the new default is awkward, but this is a starting point.

[bgilbert]

OpenSSH 8.2 will add an Include directive that supports wildcards. Once this lands in Fedora, we can add Include sshd_config.d/* to the top of sshd_config and document using it to re-enable password authentication.

[bgilbert]

coreos/fedora-coreos-config#349 moved the relevant config to /etc/ssh/sshd_config.d/ in the next stream. Once that change lands in stable, we can document how to override the default and close this out.

[martinpitt]

Ugh, we just upgraded our Fedora CoreOS image, and this was a rather unexpected surprise -- all of a sudden ssh and Cockpit stopped working completely. So how does one change this back in ignition? Thanks!

[bgilbert]

@martinpitt We've been disabling password login for a long time now; the only recent change was what file we're doing it from. Had you customized your setup to re-enable it?

[martinpitt]

@bgilbert : Right, we did, just the seddery to change it in sshd_config did not work any more. (Needed to change to rm the separate file instead). Nevertheless, changing this in ignition will be cleaner in the future. Thanks!

[bgilbert]

Here's an FCC fragment:

variant: fcos
version: 1.1.0
storage:
  files:
    - path: /etc/ssh/sshd_config.d/02-enable-passwords.conf
      mode: 0644
      contents:
        inline: |
          # Fedora CoreOS disables SSH password login by default.
          # Re-enable it.
          # This file must sort before 04-disable-passwords.conf.
          PasswordAuthentication yes

[bgilbert]

Docs update in coreos/fedora-coreos-docs#80.

[dustymabe]

@martinpitt
Ugh, we just upgraded our Fedora CoreOS image, and this was a rather unexpected surprise -- all of a sudden ssh and Cockpit stopped working completely. So how does one change this back in ignition? Thanks!

Just to be clear, you aren't "upgrading" existing systems, but you updated your config and now launching new systems shows this problem? The reason I ask is I want to make sure that we don't break people's existing systems that are doing upgrades and I want to understand the problem space better.

[martinpitt]

@dustymabe: Right, sorry for being unclear there -- it's not an rpm-ostree update of an existing install, right. It's a fresh bootstrap of the latest coreos qcow image. @bgilbert , thanks for the docs update!