Migrate HLS reconciliation EC2 instance to the new CSET instance on MCP

[madhuksridhar]

Background

This EC2 instance is used to manage/deploy LPDAAC reconciliation workflow. We need to migrate this to a new CSET instance on MCP and remove the old EC2 instance.

Old instance name: HLS_reconciliation
New instance name: HLS_reconciliation_CSET_2023
LPDAAC Reconciliation repo: hls-lpdaac-orchestration

Tasks

• Set up the new instance with all dependencies
• Try to connect to the deployed Pulumi stack form the new instance.
• After successfully linking the Pulumi stack, test and ensure that the daily reconciliation procedure remains unaffected and continues to send notification reports
• Inform CMT about successful migration

[chuckwondo]

I am unable to connect via ssh. My attempt to connect times out, but I suspect that's because my ip address isn't in the allowed ranges that I see configured for inbound on port 22. I suspect these are MCP ranges:

• 35.161.197.194/32
• 207.157.81.66/32

Connecting to the VPN didn't help. However, I was able to connect via Session Manager in the browser (via AWS Console).

[chuckwondo]

The instructions for setting up prerequisites are a bit off, so I'm adding notes here for adjustment. For using AWS Session Manager:

• By default, shell is sh (at least when connecting via Session Manager), but bash should be used, so after connecting, run bash to enter a Bash shell.
• For Ubuntu, install nvm with curl -o- https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
• The preceding command adds 2 lines to the end of ~/.profile, but that file won't get sourced when switching from sh to bash, so they won't have any effect (i.e., nvm won't be found). Move the 2 lines from the end of ~/.profile to the end of ~/.bashrc
• Run exit then bash to make nvm available.
• There is no need for a step to explicitly install npm because installing node via nvm automatically installs npm as well, as follows: nvm install --lts
• Install Docker CE:

  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
  sudo apt-get update
  sudo apt-get install -y docker-ce
  docker --version
  sudo usermod -aG docker $(whoami)  # Automatically run `docker` as `sudo docker` for current user
  

• Install Pulumi: curl -fsSL https://get.pulumi.com | sh
• Add the following line to the end of ~/.bashrc:

  export PATH=${PATH}:${HOME}/.pulumi/bin
  

• Run exit then bash
• Run pulumi version to confirm pulumi can be found

This allowed me to get all prereqs installed on the instance for user ssm-user.

We may want to move Pulumi from /home/ssm-user/.pulumi to /usr/local/pulumi and add symlink from /usr/local/bin/pulumi to /usr/local/pulumi/bin/pulumi to allow other ssh users run pulumi commands (likely user ubuntu, in addition to ssm-user).

[madhuksridhar]

Compiled some notes about connecting to Pulumi stack here. Need to tagup with @chuckwondo to discuss further.