ec2_instance module will not return state when state OK (already exists)

[IPvSean]

Summary

In provisioning I am noticing that if I do

-> ec2_instance
-> do something like ec2_tag over the results of ec2_instance

it will work the first time... but ec2_instance will not return instances when it is OK, so subsequent runs will fail

However this will work everytime->

-> ec2_instance
-> ec2_instance_info
-> ec2_tag

This behavior seems wrong to me... it differs from the older ec2 module and forces the playbook author to create an additional task to do something that the ec2_instance should do by itself

Issue Type

Bug Report

Component Name

ec2_instance

Ansible Version

$ ansible --version

Collection Versions

$ ansible-galaxy collection list

➜  provisioner git:(gh-pages) ✗ ansible-galaxy collection list

# /Users/sean/.ansible/collections/ansible_collections
Collection                          Version
----------------------------------- -------
amazon.aws                          3.2.0
ansible.netcommon                   2.0.2
ansible.network                     1.2.0
ansible.posix                       1.3.0
ansible.product_demos               1.2.12
ansible.utils                       2.6.0
ansible.windows                     1.5.0
ansible.workshops                   1.0.11
arista.eos                          2.1.2
awx.awx                             19.2.2
azure.azcollection                  1.12.0
chocolatey.chocolatey               1.1.0
cisco.ios                           2.0.1
cisco.iosxr                         2.8.1
cisco.nxos                          2.9.0
community.aws                       3.2.1
community.crypto                    1.6.2
community.general                   3.0.2
community.mysql                     2.1.0
community.windows                   1.3.0
containers.podman                   1.9.3
f5networks.f5_modules               1.9.0
frr.frr                             1.0.3
junipernetworks.junos               2.1.0
openvswitch.openvswitch             2.1.0
redhat_cop.controller_configuration 2.0.0
redhat_cop.tower_utilities          0.3.2
vyos.vyos                           2.8.0

AWS SDK versions

$ pip show boto boto3 botocore

Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.20.51
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.23.51
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed
DEFAULT_FORKS(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = 50
DEFAULT_HOST_LIST(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = ['/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/hosts']
DEFAULT_STDOUT_CALLBACK(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = yaml
HOST_KEY_CHECKING(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = 60
PERSISTENT_CONNECT_TIMEOUT(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = 60
RETRY_FILES_ENABLED(/Users/sean/Documents/GitHub/IPvSean/workshops/provisioner/ansible.cfg) = False

OS / Environment

Mac OS 12.4 Monterey

Steps to Reproduce

- name: Create EC2 instances for qradar
  amazon.aws.ec2_instance:
    key_name: "{{ ec2_name_prefix }}-key"
    security_group: "{{ ec2_security_group }}"
    instance_type: "{{ ec2_info['qradar']['size'] }}"
    image_id: "{{ qradar_ami['image_id'] }}"
    region: "{{ ec2_region }}"
    exact_count: "{{ student_total }}"
    state: running
    filters:
      "tag:Workshop_qradar": "{{ ec2_name_prefix }}-qradar"
      instance-state-name: running
    tags:
      Workshop_qradar: "{{ ec2_name_prefix }}-qradar"
      Workshop: "{{ ec2_name_prefix }}"
      Workshop_type: "{{ workshop_type }}"
      AWS_USERNAME: "{{ aws_user }}"
      owner: "{{ aws_user }}"
      Info: "AWS_USERNAME that provisioned this-> {{ aws_user }}"
      Ansible_Workshops: "This was provisioned through the ansible workshops provisioner"
      Students: "{{ student_total }}"
      username: '{{ ec2_info["rhel8"]["username"] }}'
      short_name: "qradar"
      ansible-workshops: "true"
    wait: "{{ ec2_wait }}"
    network:
      assign_public_ip: true
    vpc_subnet_id: "{{ ec2_vpc_subnet_id }}"
    volumes:
      - device_name: /dev/sda1
        ebs:
          volume_type: "{{ ec2_info['qradar']['disk_volume_type'] }}"
          volume_size: "{{ ec2_info['qradar']['disk_space'] }}"
          iops: "{{ ec2_info['qradar']['disk_iops'] }}"
          throughput: "{{ ec2_info['qradar']['disk_throughput'] }}"
          delete_on_termination: true
  when: security_console == 'qradar'
  async: "{{ async_timeout_seconds }}"
  poll: 0
  register: qradar_async

- name: Ensure tags are present for qradar
  amazon.aws.ec2_tag:
    region: "{{ ec2_region }}"
    resource: "{{ item[1]['instance_id'] }}"
    state: present
    tags:
      Name: "{{ ec2_name_prefix }}-student{{ item[0] + 1 }}-qradar"
      Index: "{{ item[0] }}"
      Student: "student{{ item[0] + 1 }}"
      launch_time: "{{ item.1.launch_time }}"
      ansible-workshops: "true"
  with_indexed_items:
    - "{{ siem_output.instances }}"
  async: "{{ async_timeout_seconds }}"
  poll: 0
  register: ensure_tags_qradar_async
  when:
    - (security_console == 'qradar') and (siem_output['instance_ids'] is not none)

Expected Results

I would expect idempotency to work every-time without my own logic with the ec2_instance_info

Actual Results

TASK [../roles/manage_ec2_instances : Wait for async instance creation to complete] ********************************************************************************
ok: [localhost] => (item={'started': 1, 'finished': 0, 'ansible_job_id': '69946880772.8308', 'results_file': '/tmp/.ansible_async/69946880772.8308', 'changed': True, 'failed': False})
ok: [localhost] => (item={'started': 1, 'finished': 0, 'ansible_job_id': '430878465557.8333', 'results_file': '/tmp/.ansible_async/430878465557.8333', 'changed': True, 'failed': False})
ok: [localhost] => (item={'started': 1, 'finished': 0, 'ansible_job_id': '888423871718.8355', 'results_file': '/tmp/.ansible_async/888423871718.8355', 'changed': True, 'failed': False})
ok: [localhost] => (item={'started': 1, 'finished': 0, 'ansible_job_id': '675347621767.8377', 'results_file': '/tmp/.ansible_async/675347621767.8377', 'changed': True, 'failed': False})
ok: [localhost] => (item={'started': 1, 'finished': 0, 'ansible_job_id': '833453438550.8399', 'results_file': '/tmp/.ansible_async/833453438550.8399', 'changed': True, 'failed': False})

TASK [../roles/manage_ec2_instances : set_fact] ********************************************************************************************************************
ok: [localhost]

TASK [../roles/manage_ec2_instances : Ensure tags are present for splunk] ******************************************************************************************
skipping: [localhost]

TASK [../roles/manage_ec2_instances : Ensure tags are present for qradar] ******************************************************************************************
fatal: [localhost]: FAILED! =>
  msg: '''dict object'' has no attribute ''instances'''

TASK [../roles/manage_ec2_instances : Ensure tags are present for splunk] ******************************************************************************************
skipping: [localhost]

TASK [../roles/manage_ec2_instances : Ensure tags are present for qradar] ******************************************************************************************
fatal: [localhost]: FAILED! =>
  msg: '''dict object'' has no attribute ''instances'''

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• [plugins/modules/ec2_instance.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_instance.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @jillr @ryansb @s-hertel @tremble
click here for bot help

[abenokraitis]

👀

[jillr]

@IPvSean if I understand correctly, the scenario is that you run the playbook once to create the instances and everything works properly. If you run the playbook a second time, the ec2_instance task sees that the instances already exist so it does not act. Generally, CUD modules do not perform list or describe operations so no information about the instances is returned. Then, the ec2_tag task fails because '{{ siem_output }}' doesn't have any instance info. You have to add an ec2_instance_info task to perform a list action.

This is how the (amazon|community).aws are developed. Adding an ec2_instance_info task is the expected way to solve this.