ECS Service - KeyError: 'launchType' - _find_usage_one_cluster Method

[vincentclee]

Bug Report

Version

awslimitchecker 8.0.2

Installation Method

Jenkins w/docker plugin

docker.image("python:3.7-alpine").inside("-u root") {
    sh """
        set -o pipefail
        pip --version
        python --version
        pip install awslimitchecker
        awslimitchecker -v --no-color \
            --limit-override-json limit_overrides_us-east-1.json \
            --threshold-override-json threshold_overrides_us-east-1.json
    """
}

Supporting Software Versions

+ pip --version
pip 20.0.2 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
+ python --version
Python 3.7.6

Actual Output

INFO:awslimitchecker.connectable:Connected to ecs in region us-east-1
Traceback (most recent call last):
  File "/usr/local/bin/awslimitchecker", line 8, in <module>
    sys.exit(console_entry_point())
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/runner.py", line 548, in console_entry_point
    r.console_entry_point()
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/runner.py", line 520, in console_entry_point
    res, problems, problem_str = self.check_thresholds(metrics)
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/runner.py", line 334, in check_thresholds
    service=self.service_name
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/checker.py", line 633, in check_thresholds
    tmp = cls.check_thresholds()
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/services/base.py", line 257, in check_thresholds
    self.find_usage()
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/services/ecs.py", line 73, in find_usage
    self._find_usage_clusters()
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/services/ecs.py", line 115, in _find_usage_clusters
    self._find_usage_one_cluster(cluster['clusterName'])
  File "/usr/local/lib/python3.7/site-packages/awslimitchecker/services/ecs.py", line 139, in _find_usage_one_cluster
    if svc['launchType'] != 'EC2':
KeyError: 'launchType'

I can provide the double verbose logs internally.

Expected Output

INFO:root:Finished checking limits in 1 seconds

TrustedAdvisor

Enterprise

Testing Assistance

Yes, i will be able to assist in testing pre-release code for the feature.

Thank You Jason.

[jantman]

This appears to happen on any ECS service that specifies Capacity Providers. In that case, the service no longer has a launchType. This also means that we're going to either need to parse and make sense of the capacity provider configuration (which looks really painful) or think about just using a CloudWatch metric for this, if possible...

[jantman]

I'm beginning to finally look into this; apologies for the delay.

• Unfortunately, CloudWatch only provides metrics for CPU / Memory utilization and reservation. Nothing useful for this use case.
• Service Quotas doesn't provide usage information for this, at least not reliably and with the granularity we need.
• The only remaining option, then, is to figure out the capacity provider configuration...

[jantman]

Ok, it appears that there have been some major changes to the ECS service limits/quotas that might make this quite a bit easier.

According to changes made to the ECS Service Quotas documentation on August 24th of this year ( awsdocs/amazon-ecs-developer-guide@3ba9bc2#diff-d98743b56c4036e0baeb5e15901d2a73 ), there is no longer a "EC2 Tasks per Service (desired count)" ("Tasks using the EC2 launch type per service (the desired count)") limit; there is now just a "Tasks per service" limit.

In addition (to complicate things a bit) there is no longer a "Fargate Tasks" limit, but rather a "Fargate On-Demand resource count" limit with a description of "The maximum number of Amazon ECS tasks and Amazon EKS pods running concurrently on Fargate in this account in the current Region." as well as a "Fargate Spot resource count" with a description of "The maximum number of Amazon ECS tasks running concurrently on Fargate Spot in this account in the current Region." These limits do, however, have CloudWatch metrics to retrieve their usage, which might be the best way to do that.

[jantman]

I've just gotten confirmation from AWS Support that the new "Tasks per service" limit includes both EC2 and Fargate tasks.

[jantman]

This has been fixed in 9.0.0, which is now live on PyPI and on the Docker Hub. Thank you so much!

[vincentclee]

This is great news! Thanks @jantman