Unable to run Vault agent as a Windows service

[mkisielewski]

Describe the bug
Unable to run Vault agent as a Windows service using builtin mechanism (sc.exe). Vault agent run in background, perform authentication using AppRole and fetch token into the file, but service return error:

PS C:\Program Files\vault> sc.exe start "Vault Agent"
[SC] StartService FAILED 1053:

The service did not respond to the start or control request in a timely fashion.

Vault agent can be also started manually using the same command as specified in service definition.

PS C:\Program Files\vault> sc.exe qc "Vault Agent"
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: Vault Agent
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : "C:/Program Files/vault/vault.exe" agent -config "C:/Program Files/vault/vault-agent.hcl"
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Vault Agent
        DEPENDENCIES       :
        SERVICE_START_NAME : LocalSystem

Probably Vault's Windows exe doesn't report status required by Windows Service Control Manager like i.e. Consul.

To Reproduce
Steps to reproduce the behavior:

1. Define Windows service using i.e. Ansible:

  - name: Create Vault agent service
    win_service:
      name: "Vault Agent"
      path: "\"{{ vault_config_dir }}/vault.exe\" agent -config \"{{ vault_config_dir }}/vault-agent.hcl\""
      description: "HashiCorp Vault Agent"
      start_mode: auto
      state: stopped

2. Run PS C:\Program Files\vault> sc.exe start "Vault Agent"
3. See error

[SC] StartService FAILED 1053:

The service did not respond to the start or control request in a timely fashion.`

Expected behavior
Be able to run Vault agent as a Windows service like in case of Consul:

PS C:\Program Files\vault> sc.exe start Consul

SERVICE_NAME: Consul
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x7d0
        PID                : 844
        FLAGS              :
PS C:\Program Files\vault> sc.exe query Consul

SERVICE_NAME: Consul
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

Environment:

• Vault Server Version: 1.2.2
• Vault Agent Version: 1.2.2
• Server Operating System/Architecture: Microsoft Windows Server 2012 R2 Standard ( 6.3.9600 N/A Build 9600)

Vault agent configuration file(s):

vault {
  address = "https://vault.service.consul:8200"
  ca_cert = "C:/Program Files/vault/tls/###.pem"
  client_cert = "C:/Program Files/vault/tls/###.crt.pem"
  client_key = "C:/Program Files/vault/tls/###.key.pem"
}

auto_auth {
    method "approle" {
    mount_path = "auth/approle"
    config = {
      role_id_file_path = "C:/Program Files/vault/role_id"
      secret_id_file_path = "C:/Program Files/vault/secret_id"
      remove_secret_id_file_after_reading = false
    }
  }
    sink "file" {
    config = {
      path = "C:/Program Files/vault/vault-token-via-agent"
    }
  }
}

[iancward]

I can confirm that this affects both Windows Server 2012 R2 as well as Windows Server 2019.
The error comes up almost immediately on 2012; however, it takes a bit for it to show up on 2019, but it does.

[mkisielewski]

I can confirm that this affects both Windows Server 2012 R2 as well as Windows Server 2019.
The error comes up almost immediately on 2012; however, it takes a bit for it to show up on 2019, but it does.

As a workaround you can use NSSM - the Non-Sucking Service Manager to manage Vault Agent running as a service under Windows.

[GrononS]

It doesn't make sense to require Windows Server users to have to log onto the server and manually launch vault.exe. Then expect us to leave our account logged in so the program doesn't close. This is just not feasible. We need the option to run as a service.

[Ankurkh1]

Any update on this please? We can already run Consul and Nomad as a native windows service. Why is Vault lagging behind?

[raskchanky]

Apologies for the radio silence on this issue. Up until now, this work just hasn't been prioritized. I'm looking into what it would take, though, and hope to have a more meaningful update sometime soon.

[iancward]

Not sure if this is related, but if I put a vault read call in a PowerShell script, and have $ErrorActionPreference="Stop", and vault encounters an error ("missing client token" is what I've tested with), PowerShell seems to be completely clueless to the fact that the command failed, and the whole script just keeps humming along like nothing is wrong.

[raskchanky]

@iancward I'm not sure I'm following your point as it relates to running the agent as a windows service. Can you elaborate a bit?

[raskchanky]

Just to follow up here for the people interested, this work has merged and should be available in 1.7.

[PhillRogers]

I'm testing v1.7.3 and the service immediately stops after being started. No useful detail in the event viewer.

[MrFixit96]

I've verified that Vault v1.7.3 works on Windows Server 2019 using the command "C:\hashicorp\vault\vault.exe" agent -config="C:\hashicorp\vault\agent-config.hcl"

However, Vault Server is still not able to be ran as a service... The commits that added this feature to Vault Agent will need to be applied in a similar fashion for Vault Server