Bug: sam sync - InvalidSamDocumentException #5432

CoshUS · 2023-06-30T00:25:09Z

Description:

sam sync --watch does not handle InvalidSamDocumentException gracefully during infra sync.

Steps to reproduce:

Run sync watch on a template containing AWS::Serverless::Api and has StageName specified.
Wait for sync watch to complete initial infra sync
Update template and remove StageName from template
Save changes and trigger infra sync
Watch errors out and stops execution with InvalidSamDocumentException

Observed result:

Code trigger

sam sync --watch --no-dependency-layer

The SAM CLI will use the AWS Lambda, Amazon API Gateway, and AWS StepFunctions APIs to upload your code without
performing a CloudFormation deployment. This will cause drift in your CloudFormation stack.
**The sync command should only be used against a development stack**.

Queued infra sync. Waiting for in progress code syncs to complete...
Starting infra sync.
Manifest is not changed for (IngestionFunction, VersionCheckFunction), running incremental build
Building codeuri: E:\...\lambda runtime: python3.8 metadata: {} architecture: x86_64 functions: IngestionFunction, VersionCheckFunction
Running PythonPipBuilder:CopySource
Running PythonPipBuilder:CopySource

Build Succeeded

Successfully packaged artifacts and wrote output template to file C:\Users\oo0SP\AppData\Local\Temp\tmpzonf0myr.
Execute the following command to deploy the packaged template
sam deploy --template-file C:\Users\...\AppData\Local\Temp\tmpzonf0myr --stack-name <YOUR STACK NAME>

Template haven't been changed since last deployment, skipping infra sync...
CodeTrigger not created as CodeUri or DefinitionUri is missing for ApiGatewayApi.
Skipped infra sync as the local template is in sync with the cloud template.
Required code syncs are queued up.
Syncing Lambda Function IngestionFunction...
Syncing Lambda Function VersionCheckFunction...
Finished syncing Lambda Function VersionCheckFunction.
Finished syncing Lambda Function IngestionFunction.
Queued infra sync. Waiting for in progress code syncs to complete...
Starting infra sync.
Failed to sync infra. Code sync is paused until template/stack is fixed.
Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\samlib\wrapper.py", line 73, in run_plugins
    parser.parse(template_copy, all_plugins)  # parse() will run all configured plugins
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\samlib\wrapper.py", line 144, in parse
    raise InvalidDocumentException(document_errors)
samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('ApiGatewayApi', "Missing required property 'StageName'.")]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 248, in _execute_infra_sync
    infra_sync_result = self._execute_infra_context(first_sync)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 178, in _execute_infra_context
    return self._infra_sync_executor.execute_infra_sync(first_sync)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\infra_sync_executor.py", line 167, in execute_infra_sync
    self._build_context.set_up()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\build\build_context.py", line 190, in set_up
    self._stacks, remote_stack_full_paths = SamLocalStackProvider.get_stacks(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_stack_provider.py", line 255, in get_stacks
    current = SamLocalStackProvider(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_stack_provider.py", line 51, in __init__
    self._template_dict = self.get_template(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_base_provider.py", line 188, in get_template
    template_dict = SamTranslatorWrapper(template_dict, parameter_values=parameters_values).run_plugins()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\samlib\wrapper.py", line 75, in run_plugins
    raise InvalidSamDocumentException(
samcli.commands.validate.lib.exceptions.InvalidSamDocumentException: [InvalidResourceException('ApiGatewayApi', "Missing required property 'StageName'.")] ('ApiGatewayApi', "Missing
required property 'StageName'.")

Error: [InvalidResourceException('ApiGatewayApi', "Missing required property 'StageName'.")] ('ApiGatewayApi', "Missing required property 'StageName'.")
Traceback:
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\metric.py", line 184, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\metric.py", line 149, in wrapped
    return_value = func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\event.py", line 350, in wrapped
    raise exception
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\event.py", line 342, in wrapped
    return_value = func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\cli_validation\image_repository_validation.py", line 92, in wrapped
    return func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\metric.py", line 103, in wrapped
    return func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\utils\version_checker.py", line 42, in wrapped
    actual_result = func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\cli\main.py", line 95, in wrapper
    return func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\_utils\cdk_support_decorators.py", line 39, in wrapped
    return func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\_utils\command_exception_handler.py", line 88, in wrapper_command_exception_handler
    raise ex
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\_utils\command_exception_handler.py", line 68, in wrapper_command_exception_handler
    return func(*args, **kwargs)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\sync\command.py", line 212, in cli
    do_cli(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\sync\command.py", line 381, in do_cli
    execute_watch(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\sync\command.py", line 539, in execute_watch
    watch_manager.start()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 209, in start
    self._start()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 224, in _start
    self._execute_infra_sync(first_sync)
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 259, in _execute_infra_sync
    self._add_template_triggers()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\sync\watch_manager.py", line 151, in _add_template_triggers
    stacks = SamLocalStackProvider.get_stacks(self._template)[0]
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_stack_provider.py", line 255, in get_stacks
    current = SamLocalStackProvider(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_stack_provider.py", line 51, in __init__
    self._template_dict = self.get_template(
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\providers\sam_base_provider.py", line 188, in get_template
    template_dict = SamTranslatorWrapper(template_dict, parameter_values=parameters_values).run_plugins()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\samlib\wrapper.py", line 75, in run_plugins
    raise InvalidSamDocumentException(

An unexpected error was encountered while executing "sam sync".

Expected result:

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: Windows 10
sam --version: 1.89.0
AWS region: us-west-2

{
  "version": "1.89.0",
  "system": {
    "python": "3.8.8",
    "os": "Windows-10-10.0.19041-SP0"
  },
  "additional_dependencies": {
    "docker_engine": "Not available",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Add --debug flag to command you are running

The text was updated successfully, but these errors were encountered:

CoshUS · 2023-06-30T00:28:22Z

One solution might be using existing stack.location for creating TemplateTrigger when templates are invalid instead of parsing the template and getting new locations.

aws-sam-cli/samcli/lib/sync/watch_manager.py

Line 153 in 30336bc

template = stack.location

hnnasit · 2023-07-01T22:16:41Z

Hi @CoshUS, thanks for reporting the issue and providing a possible solution. I was able to reproduce this behavior, marking it as a bug.

hnnasit · 2023-07-03T20:39:27Z

Hey Wilton, feel free to raise a PR with the fix if you are interested in contributing 😅.

github-actions · 2023-07-27T18:52:16Z

Patch is released in v1.94.0. Closing

CoshUS added the stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. label Jun 30, 2023

hnnasit added type/bug area/sync sam sync command and removed stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. labels Jul 1, 2023

hnnasit added the contributors/welcome Contributors are welcome to work on this label Jul 3, 2023

mndeveci mentioned this issue Jul 14, 2023

fix: don't run sam transform when getting (nested) stack information #5527

Merged

8 tasks

mndeveci added the stage/waiting-for-release Fix has been merged to develop and is waiting for a release label Jul 21, 2023

github-actions bot closed this as completed Jul 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: sam sync - InvalidSamDocumentException #5432

Bug: sam sync - InvalidSamDocumentException #5432

CoshUS commented Jun 30, 2023

CoshUS commented Jun 30, 2023

hnnasit commented Jul 1, 2023

hnnasit commented Jul 3, 2023

github-actions bot commented Jul 27, 2023

Bug: sam sync - InvalidSamDocumentException #5432

Bug: sam sync - InvalidSamDocumentException #5432

Comments

CoshUS commented Jun 30, 2023

Description:

Steps to reproduce:

Observed result:

Expected result:

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

CoshUS commented Jun 30, 2023

hnnasit commented Jul 1, 2023

hnnasit commented Jul 3, 2023

github-actions bot commented Jul 27, 2023