Skip to content

Windows Worker and Concourse Setup

Jump to bottom
Amin Jamali edited this page Aug 16, 2018 · 7 revisions

Windows Worker Setup

Instructions for setting up a Windows worker for building vSphere stemcells and optionally registering it with Concourse.

Host machine requirements

  1. Windows Server 2012R2 or Windows Server 2016
  2. 8GB RAM minimum / 12GB recommended
  3. 6 CPU cores minimum / 8 recommended
    • If using 8 or less cores you will want to set the NUM_VCPUS environment variable when invoking the build rake task.
  4. 200GB of disk space minimum / 500GB recommended
  5. If the host is a virtual machine it must have hardware assisted virtualization:
    • VMware vSphere/ESXi: VM Hardware => Virtual Hardware => CPU => Expose hardware assisted virtualization to the guest OS
    • VMware Fusion: Settings => Processors & Memory => Advanced options => Enable hypervisor applications in this virual machine
    • VMware Workstation: Virtual Machine Settings => Processors => Virtualize Intel VT-x/EPT or AMD-V/RVI

Software Requirements

Run stemcell_builder_setup.ps1 to skip this section.

  1. Golang (latest)
  2. Ruby 2.3.X
  3. tar.exe
  4. Packer: Download the Windows binary from the linked release.
  5. VMware Workstation 12.5 Pro: A free trial is available, but will expire in 30 days

Concourse specific

  1. concourse_windows_amd64.exe
  2. WinSW v2.x.x: Download

Installation

Run stemcell_builder_setup.ps1 to skip this section.

Installers

  1. Golang: use defaults
  2. Ruby: in the Installation Destination and Optional Tasks section (immediately after license agreement) select: "Add Ruby executables to your PATH" and "Associate .rb and .rbw files with Ruby installation". 
    gem install bundler
  3. VMware Workstation: use defaults, you can select the "Enhanced Keyboard Driver" if desired.
    • If VT-x (Intel hardware virtualization) pass-through is not enabled you may get an error during the install or when trying to run Workstation. Instructions are available in the Host machine requirements section.
  4. Add ovftool to the System path using the below PowerShell script: 
    $ovftool="C:\Program Files (x86)\VMware\VMware Workstation\OVFTool"
[Environment]::SetEnvironmentVariable("Path", "$env:PATH;$ovftool", [System.EnvironmentVariableTarget]::Machine)

Executables

  1. Create a C:\bin directory and add it to the System PATH: 
    New-Item -ItemType directory -Path C:\bin -Force
[Environment]::SetEnvironmentVariable("Path", "$env:PATH;C:\bin", [System.EnvironmentVariableTarget]::Machine)
  2. Tar: Copy the downloaded tar executable to C:\bin\tar.exe - you may have to rename the downloaded file.
  3. Packer: Extract the downloaded packer zip and copy packer.exe to C:\bin\packer.exe

Check that everything is installed

  1. Run the below script to make sure everything is installed: 
    $RequiredExes=@(
    "tar.exe",
    "packer.exe",
    "ovftool.exe",
    "go.exe",
    "ruby.exe"
)
foreach ($exe in $RequiredExes) {
    Get-Command $exe
}

Setup VMX Directory and Initial VM

The initial VM that will be used by the build process needs to be created by hand.

  1. Follow the documentation for creating a base VM: here.
    • Note:
      1. Follow only the steps listed in: 'Step 1: Create base VM for stemcell'
      2. Name the VM vmx-1 and note where it is saved
  2. Turn on WinRM in the base VM:
    • Load the code found here in PowerShell and run Enable-WinRM
  3. When done shutdown the VM.
  4. Find the directory where the VM is saved
    • The default location is %USERPROFILE%\Documents\Virtual Machines
    • If you named your VM vmx-1 it is likely located at %USERPROFILE%\Documents\Virtual Machines\vmx-1
  5. Create the directory: C:\vmx-data
  6. Create a gzip'd tar archive of the base VM and save it to C:\vmx-data\vmx-1.tgz, below is a helper script: 
    # Stop on error
$ErrorActionPreference = "Stop";

# Directory where you VM is saved
$VMDir="$env:USERPROFILE\Documents\Virtual Machines"

# Name of your virtual machine - must be vmx-1
$VMName="vmx-1"

# Tarball we will create
$Tarball="C:\vmx-data\vmx-1.tgz"

if ($VMName -ne "vmx-1") {
    Write-Error "Invalid VM name: $VMName - rename it's directory to vmx-1"
}
if (-Not (Test-Path "$VMDir\$VMName")) {
    Write-Error "Invalid VM directory and name: $VMDir\$VMName"
}
if (Test-Path $Tarball) {
    Write-Error "Error: tarball already exists - refusing to overwrite: $Tarball"
}
if (-Not (Test-Path "C:\vmx-data")) {
    New-Item -ItemType directory -Path "C:\vmx-data"
}
Push-Location $VMDir
    tar.exe czvf "$Tarball" "$VMName"
    if ($LASTEXITCODE -ne 0) {
        Write-Error "Tar exited with: $LASTEXITCODE"
    }
Pop-Location

Note: If you are not using Concourse the setup is complete.

Install Concourse Worker

This section documents how to configure the worker for Concourse. If you are not using Concourse the setup of your worker is complete.

Concourse Setup (assumes Concourse is deployed with BOSH):

  1. Make sure your Concourse deployment is configured to support external workers: Supporting external workers
    • This requires manually configuring the TSA'a host key
  2. Generate the worker's RSA keys: ssh-keygen -t rsa -b 4096 -N "" -f tsa-worker-private-key
    • ssh-keygen is available on macOS, Linux and can be installed on Windows with Git for Windows
  3. Add the workers public key tsa-worker-private-key.pub to the authorized_keys property of the tsa job in your Concourse deployment manifest.

Worker Setup:

  1. Create the following directories: C:\containers, C:\concourse and C:\vmx-data

    foreach ($name in @("containers", "concourse", "vmx-data")) {
    if (-Not (Test-Path "C:\$name")) {
        New-Item -ItemType directory -Path "C:\$name" -Force
    }
}

  2. Move concourse_windows_amd64.exe to C:\concourse\concourse_windows_amd64.exe

  3. Move WinSW.NET4.exe to C:\concourse\concourse.exe (renaming to it concourse.exe)

  4. Save the generated tsa-worker-private-key to C:\concourse\tsa-worker-private-key

  5. Save the TSA hosts public key to C:\concourse\tsa-public-key.pub

  6. Save below configuration as C:\concourse\concourse.xml replacing the TSA_HOST_ADDRESS and WORKER_TAG_NAME fields

    • TSA_HOST_ADDRESS: The address of your TSA host: 10.0.0.6 or my-ci.my-app.com
    • WORKER_TAG_NAME: The tag you will use to specify this worker in your pipeline, for example "windows-worker"
    <service>
  <id>concourse</id>
  <name>Concourse</name>
  <description>Concourse Windows worker.</description>
  <startmode>Automatic</startmode>
  <executable>C:\concourse\concourse_windows_amd64.exe</executable>
  <argument>worker</argument>
  <argument>/work-dir</argument>
  <argument>C:\containers</argument>
  <argument>/tsa-worker-private-key</argument>
  <argument>C:\concourse\tsa-worker-private-key</argument>
  <argument>/tsa-public-key</argument>
  <argument>C:\concourse\tsa-public-key.pub</argument>
  <argument>/tsa-host</argument>
  <argument>TSA_HOST_ADDRESS</argument> <!-- REPLACE ME -->
  <argument>/tag</argument>
  <argument>WORKER_TAG_NAME</argument> <!-- REPLACE ME -->
  <onfailure action="restart" delay="10 sec"/>
  <onfailure action="restart" delay="20 sec"/>
  <logmode>rotate</logmode>
</service>

  7. Make sure everything is in the right place:

    # Required directories
$Failed=$false
foreach ($dir in @("C:\containers", "C:\concourse", "C:\vmx-data")) {
    if (-Not (Test-Path $dir)) {
        Write-Host -ForegroundColor Red "Error: Missing required directory: $dir"
        $Failed=$true
    }
}
$RequiredFiles=@(
    "concourse_windows_amd64.exe",
    "concourse.exe",
    "concourse.xml",
    "tsa-worker-private-key",
    "tsa-public-key.pub"
)
foreach ($name in $RequiredFiles) {
    $path = "C:\concourse\$name"
    if (-Not (Test-Path $path)) {
        Write-Host -ForegroundColor Red "Error: missing required file: $path"
        $Failed=$true
    }
}
# Make sure TSA_HOST_ADDRESS and WORKER_TAG_NAME were replaced
$Config="C:\concourse\concourse.xml"
if (Test-Path $Config) {
    $Content=(Get-Content -Raw -Path $Config)
    if ($Content.Contains("TSA_HOST_ADDRESS")) {
        Write-Host -ForegroundColor Red "Error: '$Config' contains: TSA_HOST_ADDRESS"
        $Failed=$true
    }
    if ($Content.Contains("WORKER_TAG_NAME")) {
        Write-Host -ForegroundColor Red "Error: '$Config' contains: WORKER_TAG_NAME"
        $Failed=$true
    }
}
if ($Failed) {
    Write-Host -ForegroundColor Red "Error: check failed"
} else {
    Write-Host -ForegroundColor Green "Success: check passed"
}

  8. Install concourse service: C:\concourse\concourse.exe install

  9. Start the concourse service C:\concourse\concourse.exe start

    • This will also register the worker with your Concourse deployment

  10. Make sure the service is running:

    • Service wrapper: C:\concourse\concourse.exe status
    • PowerShell: (Get-Service -Name concourse).Status
    • Use the Task Manager or Services GUIs

  11. Make sure the the worker is registered with your Concourse: fly -t TARGET workers

Note: You must restart the service for changes to concourse.xml to be applied: C:\concourse\concourse.exe restart

Debugging Concourse Worker

  1. Check The service logs:
    • Stderr: C:\concourse\concourse.err.log
    • Stdout: C:\concourse\concourse.out.log
    • Service Wrapper: C:\concourse\concourse.wrapper.log
  2. Make sure the workers public key has been added to the TSA hosts authorized_keys
Clone this wiki locally