Translator Wrapper defaults to `us-east-1` for SAR applications #2065

natasha-audrey · 2020-06-23T00:53:28Z

Description

sam build does not replace the region properly when transforming the template. The region is defined as us-east-1 and that is the region used on transform in SamTranslatorWrapper regardless of AWS_DEFAULT_REGION, .aws/config, or region flag values

Steps to reproduce

Reproduction repo: https://github.com/NathanYocum/sam-sar-bug (requires changing account id's and an s3 bucket in us-east-1 and us-east-2).

pip install aws-sam-cli==0.53.0
Publish a SAR application to us-east-1 and us-east-2
Build or deploy a template that uses the SAR application with the following for example:

# template.yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Mappings:
  SAMBuildTest:
    us-east-1:
      # Replace AccountId
      ApplicationId: arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test
      SemanticVersion: 0.0.2
    us-east-2:
      # Replace AccountId
      ApplicationId: arn:aws:serverlessrepo:us-east-2:854484851566:applications/sam-build-test
      SemanticVersion: 0.0.2
Resources:
  SAMBuildSAR:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: !FindInMap [ SAMBuildTest, !Ref AWS::Region, ApplicationId ]
        SemanticVersion: !FindInMap [ SAMBuildTest, !Ref AWS::Region, SemanticVersion ]

AWS_DEFAULT_REGION=us-east-1 sam build # passes
AWS_DEFAULT_REGION=us-east-2 sam build # fails
sam build --region us-east-2 # fails

Observed result

Command: AWS_DEFAULT_REGION=us-east-2 sam build --debug

samcli.commands.validate.lib.exceptions.InvalidSamDocumentException: [InvalidResourceException('SAMBuildSAR', 'Application with id arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test could not be found.')] ('SAMBuildSAR', 'Application with id arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test could not be found.')

Full output:

Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 556, 'exitReason': 'InvalidSamDocumentException', 'exitCode': 255, 'requestId': '35a8bf9d-ccd4-4e22-8f8c-79b6a1987a79', 'installationId': 'e7ea7dd6-88c2-49c8-80b5-45f7634ac522', 'sessionId': '31e76f7b-98d6-4bf0-90d8-74962e5c565d', 'executionEnvironment': 'CLI', 'pyversion': '3.7.7', 'samcliVersion': '0.53.0'}}]}
Telemetry response: 200
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 77, in run_plugins
    parser.parse(template_copy, all_plugins)  # parse() will run all configured plugins
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 146, in parse
    raise InvalidDocumentException(document_errors)
samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('SAMBuildSAR', 'Application with id arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test could not be found.')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/sam", line 33, in <module>
    sys.exit(load_entry_point('aws-sam-cli==0.53.0', 'console_scripts', 'sam')())
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 96, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
    return_value = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/build/command.py", line 129, in cli
    mode,
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/build/command.py", line 179, in do_cli
    mode=mode,
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/build/build_context.py", line 62, in __enter__
    self._function_provider = SamFunctionProvider(self._template_dict, self._parameter_overrides)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/providers/sam_function_provider.py", line 40, in __init__
    self.template_dict = SamFunctionProvider.get_template(template_dict, parameter_overrides)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/providers/sam_base_provider.py", line 126, in get_template
    template_dict = SamTranslatorWrapper(template_dict, parameter_values=parameters_values).run_plugins()
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 80, in run_plugins
    functools.reduce(lambda message, error: message + " " + str(error), e.causes, str(e))
samcli.commands.validate.lib.exceptions.InvalidSamDocumentException: [InvalidResourceException('SAMBuildSAR', 'Application with id arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test could not be found.')] ('SAMBuildSAR', 'Application with id arn:aws:serverlessrepo:us-east-1:854484851566:applications/sam-build-test could not be found.')

I was able to fix this by adding

        self.parameter_values["AWS::Region"] = boto3.session.Session().region_name

to https://github.com/awslabs/aws-sam-cli/blob/develop/samcli/lib/samlib/wrapper.py#L40 but I'm sure the region in context is encapsulated somewhere else better for a solution.

Expected result

Build/Deploy succeeds in us-east-2 and other regions aside from us-east-1

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS 10.15.5
sam --version: 0.53.0 (tried both the pip and brew versions to no success)

The text was updated successfully, but these errors were encountered:

timoschilling · 2020-06-23T08:51:02Z

Does is work when using --region?

sam build --region us-east-2

natasha-audrey · 2020-06-23T13:05:57Z

Does is work when using --region?
sam build --region us-east-2

It does not. I think the SAMTranslatorWrapper fills the pseudo params like AWS::Region with mock values -- but the mock value gets used to transform the ApplicationId in the on_before_transform_template action instead of a contextualized set of AWS pseudo parameters.

13125381 · 2020-07-14T10:41:45Z

I have this exact same issue on the same OS :(. Is there a fix on its way?

sriram-mv · 2020-07-15T20:31:42Z

https://github.com/awslabs/aws-sam-cli/blob/23212ef827a105d23339712adc9422c6cc55e12c/samcli/lib/providers/sam_base_provider.py#L161 seems to be the line in question.

This behavior is intentional, even with AccountId being dummy ones instead of inferencing from local creds. Is it just region that seems to be causing this issue or any other intrinsics as well?

natasha-audrey · 2020-07-16T03:30:04Z

https://github.com/awslabs/aws-sam-cli/blob/23212ef827a105d23339712adc9422c6cc55e12c/samcli/lib/providers/sam_base_provider.py#L161
seems to be the line in question.

This behavior is intentional, even with AccountId being dummy ones instead of inferencing from local creds. Is it just region that seems to be causing this issue or any other intrinsics as well?

AWS::AccountId seems like it works fine so it might just be an error with AWS::Region -- which I think the translator has some logic to transform the Resource in the on_before_transform when AWS::Region is provided

natasha-audrey · 2020-07-16T04:41:15Z

Just kidding, I modified the wrong template when manually testing, switching it to:

Resources:
  SAMBuildSAR:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: !FindInMap 
          - SAMBuildTest
          - !Ref AWS::AccountId
          - ApplicationId
        SemanticVersion: !FindInMap 
          - SAMBuildTest
          - !Ref AWS::AccountId
          - SemanticVersion

fails but with samcli.commands.validate.lib.exceptions.InvalidSamDocumentException: [InvalidResourceException('SAMBuildSAR', "Property 'ApplicationId' cannot be resolved. Only FindInMap and Ref intrinsic functions are supported.")] ('SAMBuildSAR', "Property 'ApplicationId' cannot be resolved. Only FindInMap and Ref intrinsic functions are supported.")

natasha-audrey · 2020-07-16T05:22:40Z

https://github.com/awslabs/serverless-application-model/blob/develop/samtranslator/plugins/application/serverless_app_plugin.py#L129 seems to be where the ApplicationId transformed to us-east-1 is occurring in the on_before_transform.

The dict of params passed into the translator transforms ['SAMBuildTest', {'Ref': 'AWS::Region'}, 'ApplicationId'], ['SAMBuildTest', {'Ref': 'AWS::Region'}, 'SemanticVersion'] into ['SAMBuildTest', 'us-east-1', 'ApplicationId'], ['SAMBuildTest', 'us-east-1', 'SemanticVersion'].

This works fine if I just do aws cloudformation deploy, since I think CF leaves the pseudo params unresolved until deployed, but this currently runs into issues with sam build and sam deploy. It would be ideal to just let CF do the resolution of the pseudo params but if not possible at least using the values from config/flags.

moelasmar · 2021-07-26T07:28:53Z

fix is released, I verified it using sam version 1.27.2

github-actions · 2021-07-26T07:29:23Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

natasha-audrey mentioned this issue Jun 23, 2020

serverless_app_plugin transforms ApplicationId to us-east-1 aws/serverless-application-model#1632

Closed

sriram-mv added the area/translator label Jul 20, 2020

mgrandis added area/build sam build command and removed area/translator labels Feb 9, 2021

jfuss mentioned this issue May 24, 2021

resolve pseudo region in build amd deploy comands #2884

Merged

8 tasks

moelasmar closed this as completed Jul 26, 2021

juanitosvq mentioned this issue Mar 18, 2022

Translator Wrapper defaults to us-east-1 for SAR applications #3758

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Translator Wrapper defaults to `us-east-1` for SAR applications #2065

Translator Wrapper defaults to `us-east-1` for SAR applications #2065

natasha-audrey commented Jun 23, 2020 •

edited

Loading

timoschilling commented Jun 23, 2020

natasha-audrey commented Jun 23, 2020

13125381 commented Jul 14, 2020

sriram-mv commented Jul 15, 2020

natasha-audrey commented Jul 16, 2020 •

edited

Loading

natasha-audrey commented Jul 16, 2020 •

edited

Loading

natasha-audrey commented Jul 16, 2020 •

edited

Loading

moelasmar commented Jul 26, 2021

github-actions bot commented Jul 26, 2021

Translator Wrapper defaults to us-east-1 for SAR applications #2065

Translator Wrapper defaults to us-east-1 for SAR applications #2065

Comments

natasha-audrey commented Jun 23, 2020 • edited Loading

Description

Steps to reproduce

Observed result

Expected result

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

timoschilling commented Jun 23, 2020

natasha-audrey commented Jun 23, 2020

13125381 commented Jul 14, 2020

sriram-mv commented Jul 15, 2020

natasha-audrey commented Jul 16, 2020 • edited Loading

natasha-audrey commented Jul 16, 2020 • edited Loading

natasha-audrey commented Jul 16, 2020 • edited Loading

moelasmar commented Jul 26, 2021

github-actions bot commented Jul 26, 2021

⚠️COMMENT VISIBILITY WARNING⚠️

Translator Wrapper defaults to `us-east-1` for SAR applications #2065

Translator Wrapper defaults to `us-east-1` for SAR applications #2065

natasha-audrey commented Jun 23, 2020 •

edited

Loading

natasha-audrey commented Jul 16, 2020 •

edited

Loading

natasha-audrey commented Jul 16, 2020 •

edited

Loading

natasha-audrey commented Jul 16, 2020 •

edited

Loading