This repository contains code which automatically deploys services on the compute server. By defining the infrastructure with code we prevent configuration drift, as well as improve determinism and security.
This repository consists of 2 sections, Packer is responsible for building templates of machines and operating systems. These templates are then used by Terraform for deployment. In both the Packer and Terraform section are Ansible sections, these provide fine-grained machine configuration on the application level.
This repository is validated on macOS, no guarantee can be given for other systems. In order to run the scripts present in this repo you will need the below dependencies:
- Docker
- Ansible
- Packer
- Terraform
This repository makes use of functionality from VMWare products, some of which are only accesible with a license. Make sure you have at least the following licenses on the host machines you are managing with Terraform:
- vCenter Server 7.0 Standard
- vSphere 7.0 Enterprise Plus
The services below are included, they are implemented in a modular manner so that (with some changes) they can be added or removed to the deployment as needed:
This is a dockerized Traefik instance which routes different web services based on the subdomain. Requires port 80 and 443 to be forwarded from the public domain to the IP of the service instance.
A cloud based workspace solution for standardized development environments.
A personal VPN solution so that connection to the deployment network is possible when on a remote location. Requires port 51820 to be forwarded from the public domain to the IP of the service instance.
Before starting the deployment script the .tfvars for your specific environment will need to be defined.
TODO: private key doc TODO: change le server to production doc (prevents Coder certificate error)
Executing the below command will execute in 3 sequential steps. First it will configure the vCenter and vSphere instance such that VMs can be deployed in a cluster, as well as creating an OVF content library for the packer templates. Second packer will build the templates defined in the script in parallel and store them in the content library. Third is the deployment of the services which are defined in the script:
./deploy.sh [-h] [-b]The build process of the Packer templates must be manually enabled by using the
-bor--buildflag. Be sure to set this flag when deploying for the first time, as otherwise there are no templates which can be used for the deployment of the services.
There might be issues with the deployment, or there might be another need to manually change services. Go to the terraform/deploy directory. This directory contains the current deployment configuration and can be changed by using the standard terraform commands.
You can replace individual service modules with the following command:
terraform apply -replace="module.<service>.vsphere_virtual_machine.vm"In order to change or add to the hardware configurations change or add files in the packer/hardware directory, then use these configurations in the deploy script in the root directory of the repository.
Adding, removing or changing the deployed services can be done by changing the files in the terraform/services directory, as well as the corresponding .tfvars service file in the terraform/variables directory. A service should be deployed using the ovf module defined in terraform/deploy/modules/ovf.
The following directories/files are autogenerated and should not be edited as the changes may be lost when running the script:
packer/manifeststerraform/setup/terraform.tfvarsterraform/deploy/terraform.tfvarsterraform/deploy/variables.services.tf
- Manual configuration of ISO files, OVF library and OVF permissions is required.
- Packer and Terraform run as the root user.
- Ansible playbooks are not idempotent.
- Manual key generation required
- Packer and Terraform shared variables are not DRY
Packer takes around 15 minutes for most hardware configurations. When using smaller hardware configurations the process can take up around 30 minutes. If the process does not finish after the given time consider improving the hardware specifications of your templates.
You may see an error warning about a fingerprint not matching of the remote machine when trying to start an SSH session, to resolve this warning you can execute the following command:
ssh-keygen -f ~/.ssh/known_hosts -R <address-of-host-with-warning>Big thanks to these guys, they made this repository a possibility by having standard implementations for Packer. Be sure to check them out as they have a more elaborate selection of build options. The bash script is inspired by this blog post.