(aws-ec2): ec2.InitFile.fromObject falsely converts numbers to strings

[matthias-pichler]

When creating the config file for the amazon-cloudwatch-agent using ec2.InitFile.fromObject with the below config file the startup fails with the following error:

Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp
Start configuration validation...
/opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default
2021/04/10 14:22:51 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json.tmp ...
2021/04/10 14:22:51 Invalid Json input schema.
2021/04/10 14:22:51 Under path : /agent/metrics_collection_interval | Error : Invalid type. Expected: integer, given: string
2021/04/10 14:22:51 Configuration validation first phase failed. Agent version: 1.0. Verify the JSON input is only using features supported by this version.

2021-04-10 14:22:51,669 [ERROR] Error encountered during build of logs: Command 20-start-logs-agent failed

And when reading the file on disk I get:

"agent": {"logfile": "/var/log/amazon-cloudwatch-agent.log", "metrics_collection_interval": "10"}, "metrics": {"metrics_collected": {"mem": {"measurement": ["mem_used_percent"]}, "swap": {"measurement": ["swap_used_percent"]}}, "append_dimensions": {"InstanceId": "${aws:InstanceId}", "ImageId": "${aws:ImageId}", "InstanceType": "${aws:InstanceType}"}}, "logs": {"log_stream_name": "logs", "logs_collected": {"files": {"collect_list": [{"timestamp_format": "%b %d %H:%M:%S", "file_path": "/var/log/gitlab-runner.log", "log_group_name": "gitlab-ci-runner-ManagerLogGroup031FB07B-880ZIAOH7NBU", "log_stream_name": "gitlab-runner"}, {"file_path": "/var/log/amazon-cloudwatch-agent.log", "log_group_name": "gitlab-ci-runner-ManagerLogGroup031FB07B-880ZIAOH7NBU", "log_stream_name": "amazon-cloudwatch-agent"}, {"timestamp_format": "%Y-%m-%d %H:%M:%S,%f", "file_path": "/var/log/cfn-init-cmd.log", "log_group_name": "gitlab-ci-runner-ManagerLogGroup031FB07B-880ZIAOH7NBU", "log_stream_name": "cfn-init-cmd"}, {"timestamp_format": "%Y-%m-%d %H:%M:%S,%f", "file_path": "/var/log/cfn-init.log", "log_group_name": "gitlab-ci-runner-ManagerLogGroup031FB07B-880ZIAOH7NBU", "log_stream_name": "cfn-init"}]}}}}

Notice how metrics_collection_interval is now a string as opposed to number in the actual code.

Reproduction Steps

Use ec2.InitFile.fromObject with a file that contains a number and observe how it is written to disk with a string instead.

What did you expect to happen?

Types of JSON objects should be preserved, it might be required to simply perform a JSON.stringify inside ec2.InitFile.fromObject and pass it through to ec2.InitFile.fromString

What actually happened?

I think the culprit might be cfn-init here which turns all values into strings 🤷

Environment

• CDK CLI Version: 1.97.1
• Framework Version: 1.97.1
• Node.js Version: 15.13.0
• OS: MacOS
• Language (Version): TypeScript (4.2.3)

Other

{
    agent: {
      metrics_collection_interval: 10,
      logfile: "/var/log/amazon-cloudwatch-agent.log",
    },
    logs: {
      log_stream_name: "logs",
      logs_collected: {
        files: {
          collect_list: [
            {
              file_path: "/var/log/gitlab-runner.log",
              log_group_name: props.logGroup.logGroupName,
              log_stream_name: "gitlab-runner",
              timestamp_format: "%b %d %H:%M:%S", //Jun 30 13:50:53
            },
            {
              file_path: "/var/log/amazon-cloudwatch-agent.log",
              log_group_name: props.logGroup.logGroupName,
              log_stream_name: "amazon-cloudwatch-agent",
            },
            {
              file_path: "/var/log/cfn-init-cmd.log",
              log_group_name: props.logGroup.logGroupName,
              log_stream_name: "cfn-init-cmd",
              timestamp_format: "%Y-%m-%d %H:%M:%S,%f", //2020-06-30 14:16:35,161
            },
            {
              file_path: "/var/log/cfn-init.log",
              log_group_name: props.logGroup.logGroupName,
              log_stream_name: "cfn-init",
              timestamp_format: "%Y-%m-%d %H:%M:%S,%f", //2020-06-30 14:16:35,161
            },
          ],
        },
      },
    },
    metrics: {
      append_dimensions: {
        /* eslint-disable no-template-curly-in-string */
        ImageId: "${aws:ImageId}",
        InstanceId: "${aws:InstanceId}",
        InstanceType: "${aws:InstanceType}",
        /* eslint-enable no-template-curly-in-string */
      },
      metrics_collected: {
        mem: {
          measurement: ["mem_used_percent"],
        },
        swap: {
          measurement: ["swap_used_percent"],
        },
      },
    },
  }

---

This is 🐛 Bug Report

[rix0rrr]

I think the culprit might be cfn-init here which turns all values into strings 🤷

I think you are probably right.

Calling stack.toJsonString() on the value, and foregoing the built-in facility of cfn-init to stringify JSON objects, is probably the right way to fix this. Nested {Ref}s etc. would (probably) still work, at the expense of the JSON file ending up a little unreadable.

This would need experimentation though.

[deybhayden]

Hey everyone, I just stubbed my toe one this issue installing amazon-cloudwatch-agent as well. Curious if anyone got around this and still used ec2.InitFile.fromObject(), or if people just use ec2.InitFile.fromString() instead.

[arewa]

Hey guys,

I tested described functionality and could not reproduce the issue with the latest version of cdk (commit 715158f).

Decided to update appropriate unit and integration tests for ec2.InitFile.fromObject to cover int, bool, number types as well as string.

@matthias-pichler-warrify, if you still facing this issue, could you please mention your aws-cdk version?
I would verify with it then.

Link to PR: #21394

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.