Azure: Filter Configuration - Need Examples

[aakifshaikh]

1. I just came across this filter called CONFIGURATION. I am not sure how to use it. There's no example within the documentation or Github. I am excited to see its potential use- does it refer to the "settings/configurations" for the supported resources. If that's the case, can you please give some examples (let's say- storage account- how can we find all those configurations then?)

https://cloudcustodian.io/docs/azure/resources/azure-common-filters.html#azure-common-filters-configuration

Can you please share with me a repo where you may have written a lot of azure policies?

2. Is FunctionApp supported by CloudCustodian? what is the resource: azure.functionapp?
3. Does it support tag at subscription level- I don't see it in the filter and action?

[aakifshaikh]

Looks like I am using the correct resource type, filter, and action and the custodian is still failing the validation. I am having the same issue with storageaccount-TLS-not-latest-version.yml

Policy

policies:

- name: storageaccount-TLS-not-latest-version
  resource: azure.storage
  description: |
    Find all storage accounts which are not using the latest TLS version (v1.2).
    This policy is running every.
  filters:
    - type: configuration
      key: minimumTlsVersion
      op: ne
      value: '1.2'
  mode:
    type: azure-periodic
    schedule: 0 */10 * * * *
    provision-options:
      identity:
        type: UserAxxxxxxxxned
        id: x
    execution-options:
      output_dir: azure://cloudcuxxxxxxxm/%d/%H/}
  actions:
    - type: logic-app
      resource-group: cloudxxxxx            
      logic-app-name: custodianxxxx    
      batch: false                               
      body: >                                    
          {
          PolicyName: policy.name,
          PolicyDescription: policy.description,
          Resource: resource.
              {
              Name: name,
              Location: location,
              Owner: tags.owner,
              VmSize: properties.hardwareProfile.vmSize
              }
          }

ERROR in Validation

2021-03-12 12:40:48,078: custodian.commands:ERROR Configuration invalid: sec-n-storageaccount-TLS-not-latest-version.yml
2021-03-12 12:40:48,079: custodian.commands:ERROR {'name': 'sec-n-storageaccount-TLS-not-latest-version', 'resource': 'azure.storage', 'description': 'Find all storage accounts which are not using the latest TLS version (v1.2).\nThis policy is running every.\n', 'filters': [{'type': 'configuration', 'key': 'minimumTlsVersion', 'op': 'ne', 'value': '1.2'}], 'mode': {'type': 'azure-periodic', 'schedule': '0 */10 * * * *', 'provision-options': {'identity': {'type': 'UserAssigned', 'id': '000xxxxxxx'}}, 'execution-options': {'output_dir': 'azure://cloudcusxxxxxxxxxxe}/{now:%Y/%m/%d/%H/}'}}, 'actions': [{'type': 'logic-app', 'resource-group': 'cloud-custodian', 'logic-app-name': 'custodianxxxxxx', 'batch': False, 'body': '{ PolicyName: policy.name, PolicyDescription: policy.description, Resource: resource.\n    {\n    Name: name,\n    Location: location,\n    Owner: tags.owner,\n    VmSize: properties.hardwareProfile.vmSize\n    }\n}'}]} is not valid under any of the given schemas

Failed validating 'anyOf' in schema['properties']['policies']['items']:
    {'anyOf': [{'$ref': '#/definitions/resources/azure.storage/policy'}]}

On instance['policies'][0]:
    {'actions': [{'batch': False,
                  'body': '{ PolicyName: policy.name, PolicyDescription: '
                          'policy.description, Resource: resource.\n'
                          '    {\n'
                          '    Name: name,\n'
                          '    Location: location,\n'
                          '    Owner: tags.owner,\n'
                          '    VmSize: properties.hardwareProfile.vmSize\n'
                          '    }\n'
                          '}',
                  'logic-app-name': 'custodianxxxx',
                  'resource-group': 'cloudxxxx',
                  'type': 'logic-app'}],
     'description': 'Find all storage accounts which are not using the '
                    'latest TLS version (v1.2).\n'
                    'This policy is running every.\n',
     'filters': [{'key': 'minimumTlsVersion',
                  'op': 'ne',
                  'type': 'configuration',
                  'value': '1.2'}],
     'mode': {'execution-options': {'output_dir': 'azure://cloudcuxxxxxxxxxxxxxxxY/%m/%d/%H/}'},
              'provision-options': {'identity': {'id': '000xxxxxx',
                                                 'type': 'UserAxxxed'}},
              'schedule': '0 */10 * * * *',
              'type': 'azure-periodic'},
     'name': 'sec-n-storageaccount-TLS-not-latest-version',
     'resource': 'azure.storage'}
2021-03-12 12:40:48,079: custodian.commands:ERROR 'configuration' is not one of ['cost']

Failed validating 'enum' in schema[0]['properties']['type']:
    {'enum': ['cost']}

On instance['type']:
    'configuration'

Issue with require-secure-transfer - storageaccount

(custodian) xxxx@XYZ cloud-custodian % custodian validate sec-n-blob-secure-transfer-not-enabled.yml
2021-03-12 12:39:45,865: root:ERROR specific_error failed, traceback, followed by fallback
Traceback (most recent call last):
  File "/Users/xxxxxxxxxcloud-custodian/c7n/schema.py", line 50, in validate
    resp = policy_error_scope(specific_error(errors[0]), data)
  File "/Usersxxxxxxxx/cloud-custodian/c7n/schema.py", line 122, in specific_error
    return specific_error(e)
  File "/Users/xxxxxxxxx/cloud-custodian/c7n/schema.py", line 132, in specific_error
    elif 'type' in v and t in v['properties']['type']['enum']:
KeyError: 'type'
2021-03-12 12:39:45,867: custodian.commands:ERROR Configuration invalid: sec-n-blob-secure-transfer-not-enabled.yml
2021-03-12 12:39:45,868: custodian.commands:ERROR {'name': 'sec-n-blob-secure-transfer-not-enabled', 'resource': 'azure.storage', 'description': 'Find all Storage Accounts where secure transfer has not been turned ON. \nThis policy runs every day at 9:30AM to check for the orphaned resources.\n', 'filters': [{'type': 'require-secure-transfer', 'value': False}], 'mode': {'type': 'azure-periodic', 'schedule': '0 */10 * * * *', 'provision-options': {'identity': {'type': 'UserAxxxxxxxed', 'id': '123xxxxxxxx'}}, 'execution-options': {'output_dir': 'azure://cloudcusxxxxxxxxxxm/%d/%H/}'}}, 'actions': [{'type': 'logic-app', 'resource-group': 'cloud-custodian', 'logic-app-name': 'custodianxxxx', 'batch': False, 'body': '{ PolicyName: policy.name, PolicyDescription: policy.description, Resource: resource.\n    {\n    Name: name,\n    Location: location,\n    Owner: tags.owner,\n    VmSize: properties.hardwareProfile.vmSize\n    }\n}'}]} is not valid under any of the given schemas

Failed validating 'anyOf' in schema['properties']['policies']['items']:
    {'anyOf': [{'$ref': '#/definitions/resources/azure.storage/policy'}]}

On instance['policies'][0]:
    {'actions': [{'batch': False,
                  'body': '{ PolicyName: policy.name, PolicyDescription: '
                          'policy.description, Resource: resource.\n'
                          '    {\n'
                          '    Name: name,\n'
                          '    Location: location,\n'
                          '    Owner: tags.owner,\n'
                          '    VmSize: properties.hardwareProfile.vmSize\n'
                          '    }\n'
                          '}',
                  'logic-app-name': 'custodianxxxxxxx',
                  'resource-group': 'cloudxxxxxx',
                  'type': 'logic-app'}],
     'description': 'Find all Storage Accounts where secure transfer has '
                    'not been turned ON. \n'
                    'This policy runs every day at 9:30AM to check for the '
                    'orphaned resources.\n',
     'filters': [{'type': 'require-secure-transfer', 'value': False}],
     'mode': {'execution-options': {'output_dir': 'azure://cloudcustodixxxxxxxxxxxm/%d/%H/}'},
              'provision-options': {'identity': {'id': 'xxxxx',
                                                 'type': 'UserAsxxxxxd'}},
              'schedule': '0 */10 * * * *',
              'type': 'azure-periodic'},
     'name': 'sec-n-blob-secure-transfer-not-enabled',
     'resource': 'azure.storage'}
2021-03-12 12:39:45,868: custodian.commands:ERROR 'require-secure-transfer' is not one of ['cost']

Failed validating 'enum' in schema[0]['properties']['type']:
    {'enum': ['cost']}

On instance['type']:
    'require-secure-transfer'

Policy

policies:

- name: sec-n-blob-secure-transfer-not-enabled
  resource: azure.storage
  description: |
    Find all Storage Accounts where secure transfer has not been turned ON. 
    This policy runs every dayxxxxxxxx the orphaned resources.
  filters:
    - type: require-secure-transfer
      value: false
  mode:
    type: azure-periodic
    schedule: 0 */10 * * * *
    provision-options:
      identity:
        type: UserAxxxxxd
        id: xxxxxx
    execution-options:
      output_dir: azure://cloudcustodianxxxxxxxxxm/%d/%H/}
  actions:
    - type: logic-app
      resource-group: cloudxxxxxx            
      logic-app-name: custodianxxxxxx   
      batch: false                               
      body: >                                    
          {
          PolicyName: policy.name,
          PolicyDescription: policy.description,
          Resource: resource.
              {
              Name: name,
              Location: location,
              Owner: tags.owner,
              VmSize: properties.hardwareProfile.vmSize
              }
          }

[stefangordon]

Hey @aakifshaikh - This appears to be an automated documentation generation bug. Several of the filters listed in "common filters" there aren't really common. I'm investigating it now.

The configuration filter is a special case of the generic value filter designed for use with the webapp resource. It allows you to filter on the web app configuration.

You can verify what actions and filters are available for a given resource using this syntax at the command line:

custodian schema azure.storage

If you are trying to verify the HTTPS/TLS requirement for storage then you probably want this:

policies:
  - name: storage-https
    resource: azure.storage
    filters:
    - type: value
      key: properties.supportsHttpsTrafficOnly
      op: ne
      value: true

[aakifshaikh]

@stefangordon - can you please check my filter - where I am trying to find if MFA is enabled for privileged users-

- name: az-cis-identity-mfa-not-enabled-privileged-users
  resource: azure.roleassignment
  description: |
    Find all Privileged users for whom MFA is NOT enabled.
    Enable multi-factor authentication for all user credentials who have write access to Azure resources. These include roles like
    Service Co-Administrators, Subscription Owners, and Contributors.
    This policy runs every Sunday at 8:55AM UTC (CST time: 2:55AM).
  filters:
    - type: value
      key: properties.roleName
      op: eq
      value: ['Owner','Contributors', 'admin']
    - type: value
      key: Microsoft.Online.Administration.StrongAuthenticationRequirement
      op: ne
      value: Enabled

2 Ensure that 'Python version' is the latest if used to run the web app
Not sure how can we compare the value to the latest version - any thoughts here? Similarly, I want to find versions for .Net framework, Java, Python, and PHP.

- name: az-cis-webapp-python-not-latest-version
  resource: azure.webapp
  description: |
    Find all Web Apps where Phython is NOT the latest version.
    Periodically, newer versions are released for Python software either due to security flaws or to include additional functionality. Using the latest Python version for web apps is 
    recommended in order to take advantage of security fixes, if any, and/or additional functionalities of the newer version.
    No action is required, If Python version is set to Off as Python is not used by your web app.
    This policy runs every Sunday at 8:55AM UTC (CST time: 2:55AM).
  filters:
    - type: configuration
      key: pythonVersion
      op: ne
      value: xxxxx

Thank you in advance.

[logachev]

Hi @aakifshaikh

1. I'm not sure if roleassignment has information about MFA information, I will need to look into this before I can advise.
2. Are you looking for some way to automatically detect latest instead of comparing it with Python 3.8? I don't think this is possible, I don't recall any Azure API that tells you latest available Python version for webapps.

[aakifshaikh]

These are among the controls from CIS Benchmark for Azure v1.1.0 - 02-15-2019. I am trying to write cloud custodian policies to identify the non-compliant items and report it.

1.1 Ensure that multi-factor authentication is enabled for all privileged users
1.2 Ensure that multi-factor authentication is enabled for all non-privileged users
9.6 Ensure that '.Net Framework' version is the latest, if used as a part of the web app
9.7 Ensure that 'PHP version' is the latest, if used to run the web app
9.8 Ensure that 'Python version' is the latest, if used to run the web app
9.9 Ensure that 'Java version' is the latest, if used to run the web app

CIS_Microsoft_Azure_Foundations_Benchmark_v1.1.0.pdf

[logachev]

@aakifshaikh, I will take a look today what we can do

[logachev]

@aakifshaikh For 1.1 and 1.2 I will file an issue to add the support for MFA check for users.

For 9.6-9.9, I would suggest using value filter and verify a specific version. I believe for Python, latest supported by webapps is 3.8.. I wasn't able to find any programmatic way to retrieve "latest" supported in a webapp unfortunately.

[aakifshaikh]

Thank you for working on this. can you reference those issue numbers here.

[logachev]

Just filed this for MFA: #6607