Skip to content
This repository has been archived by the owner on Aug 22, 2024. It is now read-only.

Autoscale no longer mounting /dev/xvdba #38

Closed
hyjkim opened this issue Feb 11, 2022 · 1 comment
Closed

Autoscale no longer mounting /dev/xvdba #38

hyjkim opened this issue Feb 11, 2022 · 1 comment

Comments

@hyjkim
Copy link

hyjkim commented Feb 11, 2022

Recently, my batch queues using this autoscaling script stopped working and the instances throw the following error:

+ MOUNTPOINT=/scratch
+ SIZE=200
+ VOLUMETYPE=gp3
+ DEVICE=
+ FILE_SYSTEM=btrfs
++ dirname /opt/amazon-ebs-autoscale/install.sh
+ BASEDIR=/opt/amazon-ebs-autoscale
+ . /opt/amazon-ebs-autoscale/shared/utils.sh
+ initialize
++ curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone/
+ export AWS_AZ=us-west-2b
+ AWS_AZ=us-west-2b
++ sed -e 's/[a-z]$//'
++ echo us-west-2b
+ export AWS_REGION=us-west-2
+ AWS_REGION=us-west-2
++ curl -s http://169.254.169.254/latest/meta-data/instance-id
+ export INSTANCE_ID=i-0bdd7c4f058f26739
+ INSTANCE_ID=i-0bdd7c4f058f26739
+ export EBS_AUTOSCALE_CONFIG_FILE=/etc/ebs-autoscale.json
+ EBS_AUTOSCALE_CONFIG_FILE=/etc/ebs-autoscale.json
+ PARAMS=
+ ((  4  ))
+ case "$1" in
+ FILE_SYSTEM=btrfs
+ shift 2
+ ((  2  ))
+ case "$1" in
+ MOUNTPOINT=/var/lib/docker
+ shift 2
+ ((  0  ))
+ eval set -- ''
++ set --
+ '[' '!' -z '' ']'
+ mkdir -p /usr/local/amazon-ebs-autoscale/bin /usr/local/amazon-ebs-autoscale/shared
+ cp /opt/amazon-ebs-autoscale/bin/create-ebs-volume /opt/amazon-ebs-autoscale/bin/ebs-autoscale /usr/local/amazon-ebs-autoscale/bin
+ chmod +x /usr/local/amazon-ebs-autoscale/bin/create-ebs-volume /usr/local/amazon-ebs-autoscale/bin/ebs-autoscale
+ ln -sf /usr/local/amazon-ebs-autoscale/bin/create-ebs-volume /usr/local/amazon-ebs-autoscale/bin/ebs-autoscale /usr/local/bin/
+ ln -sf /usr/local/amazon-ebs-autoscale/bin/create-ebs-volume /usr/local/amazon-ebs-autoscale/bin/ebs-autoscale /usr/bin/
+ cp /opt/amazon-ebs-autoscale/shared/utils.sh /usr/local/amazon-ebs-autoscale/shared
+ cp /opt/amazon-ebs-autoscale/config/ebs-autoscale.logrotate /etc/logrotate.d/ebs-autoscale
+ sed -e s#%%FILESYSTEM%%#btrfs#
+ sed -e s#%%VOLUMETYPE%%#gp3#
+ sed -e s#%%MOUNTPOINT%%#/var/lib/docker#
+ cat /opt/amazon-ebs-autoscale/config/ebs-autoscale.json
+ '[' -e /var/lib/docker ']'
+ '[' -d /var/lib/docker ']'
+ '[' -e /var/lib/docker ']'
+ '[' -z '' ']'
++ create-ebs-volume --size 200 --type gp3
+ DEVICE=/dev/xvdba
+ '[' btrfs = btrfs ']'
+ mkfs.btrfs -f -d single /dev/xvdba
ERROR: cannot open device '/dev/xvdba': No such file or directory
ERROR: open ctree failed
btrfs-progs v4.15.1
See http://btrfs.wiki.kernel.org for more information.

Detected a SSD, turning off metadata duplication.  Mkfs with -m dup if you want to force metadata duplication.
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
Cloud-init v. 19.3-44.amzn2 finished at Fri, 11 Feb 2022 23:23:07 +0000. Datasource DataSourceEc2.  Up 63.77 seconds

I've tried with lvm.ext4 as well and still encounter failures. If I launch the instance without the user data enabled, I can see the /dev/xvdba volume, but as soon as btrfs tries to use the volume to make the file system, I receive the error. Following the error /dev/xvdba is no longer visible.

@hyjkim
Copy link
Author

hyjkim commented Feb 11, 2022

After speaking with AWS support, it seems to be related to this issue and not with this repo: amazonlinux/amazon-ec2-utils#23

@hyjkim hyjkim closed this as completed Feb 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant