Add mapper for AWS Config Rules #68

jkufro · 2021-02-10T15:06:00Z

Adds aws_config_mapper. This pulls Ruby AWS SDK data to translate AWS Config Rule results into HDF format json to be viewable in Heimdall. Our intended use case is to incorporate this into an AWS lambda that will pull & translate config compliance and push that up to a Heimdall server via the API.

Example execution (relies on having AWS credentials set up on your local machine): heimdall_tools aws_config_mapper -m my_addl_rule_mappings.csv -o aws_config_results.json

lib/heimdall_tools/aws_config_mapper.rb

lib/heimdall_tools/help/aws_config_mapper.md

rbclark · 2021-02-10T15:21:02Z

lib/heimdall_tools/aws_config_mapper.rb

+                          when 'INSUFFICIENT_DATA'
+                            {
+                              'run_time': 0,
+                              'code_desc': INSUFFICIENT_DATA_MSG,
+                              'skip_message': INSUFFICIENT_DATA_MSG,
+                              'start_time': DateTime.now.strftime('%Y-%m-%dT%H:%M:%S%:z'),
+                              'status': 'skipped'
+                            }
+                          end


It may be worth another opinion on this, but wouldn't INSUFFICIENT_DATA mean not_reviewed and not skipped? To me insufficient data sounds like something you would need to go validate.

I chose skipped here based on the schema options here https://github.com/mitre/inspecjs/blob/master/schemas/exec-json.json#L124

Is not_reviewed also a legal option for the status field?

I did a bit more digging, not_reviewed is not a legal option, however in Heimdall an impact of 0 && a result.status of skipped is considered 'Not Applicable', and an impact of greater-than 0 and a result.status of skipped is considered 'Not Reviewed.

Just made a change to make NOT_APPLICABLE have impact set to 0

Is there a sample of AWS Config data to review? (As a former assessor, I need better context to help you decide which "bucket" to put the result in.)

I don't necessarily concur with setting whatever "insufficient data" means to "not applicable". Need some original data for context. Thanks!

rbclark · 2021-02-10T15:21:42Z

lib/heimdall_tools/aws_config_mapper.rb

+          hdf_result['status'] = case result.dig(:compliance_type)
+                                 when 'COMPLIANT'
+                                   'passed'
+                                 when 'NON_COMPLIANT'
+                                   'failed'
+                                 else
+                                   'skipped'
+                                 end


If insufficient_data is updated to map to not_reviewed then mapping here should include it as well.

aaronlippold · 2021-02-16T03:35:13Z

if possible, could we add a function to also add the impact to the control as well? Our impacts are in the range:
This will help the overall display and completeness of the data.

0.0 none
0.1 - 0.3 low ( defaults to 0.3 )
0.4 - 0.6 medium ( defaults to 0.5 )
0.7 - 0.8 high ( defaults to 0.7 )
0.9 - 1.0 critical ( defaults to 0.9 )

i would also love to see the lambda work you guys are doing - as we are also working on a lambda task - https://github.com/mitre/serverless-inspec

jkufro · 2021-02-16T03:43:27Z

if possible, could we add a function to also add the impact to the control as well? Our impacts are in the range:
This will help the overall display and completeness of the data.
0.0 none
0.1 - 0.3 low ( defaults to 0.3 )
0.4 - 0.6 medium ( defaults to 0.5 )
0.7 - 0.8 high ( defaults to 0.7 )
0.9 - 1.0 critical ( defaults to 0.9 )
i would also love to see the lambda work you guys are doing - as we are also working on a lambda task - https://github.com/mitre/serverless-inspec

@aaronlippold Are you requesting something different than the function that is defined here and is called here? I have it set up now so that you can tag an AWS Config Rule with Info, Low, Medium, High, or Critical. Should I add the ability to assign a numerical value between 0 and 1 in addition to this?

Also, totally happy to collaborate on lambda work.

rbclark · 2021-02-16T13:19:31Z

I've already reviewed this, it is just waiting on a final look from @rx294 before being merged.

rx294 · 2021-02-16T16:34:40Z

@jkufro
The current design of the aws_config_mapper requires user setup (ie tag placement) on the aws_config rules, which will be a deviation from pattern used in other mappers. Please see below suggestions to align the aws_config_mapper to the general pattern in used in heimdall_tools.

This will help with reusability and consistency of the HDFs generated by aws_config_mapper across multiple users.

Control Title mapped title of the AWS_Config rule (as is the case currently)
Description: Parsed from the aws_config rule description feild.

Check Text to include the config rule arn and parameters used for the rule. Consider formatting the data fields with linefeeds.

Set the severity to the "average" of 0.5 (medium)" for all controls.
Nist Tag: Create a CSV file mapping config_rule titles to NIST 800-53 mappings as in examples here https://github.com/mitre/heimdall_tools/blob/master/lib/data/owasp-nist-mapping.csv and use it in mapping NIST tags to the controls. This way mapping to NIST controls stays consistant and centralized at heimdall_tools codebase.
Remove references to cce_id cis_impact cis_rid fix rationale cweid confidence cis_level at this iteration.
Please add sample hdf jsons to this location https://github.com/mitre/heimdall_tools/tree/master/sample_jsons
Please add readme updates
Please resolve merge conflicts.

aaronlippold · 2021-02-17T03:08:47Z

if possible, could we add a function to also add the impact to the control as well? Our impacts are in the range:
This will help the overall display and completeness of the data.
0.0 none
0.1 - 0.3 low ( defaults to 0.3 )
0.4 - 0.6 medium ( defaults to 0.5 )
0.7 - 0.8 high ( defaults to 0.7 )
0.9 - 1.0 critical ( defaults to 0.9 )
i would also love to see the lambda work you guys are doing - as we are also working on a lambda task - https://github.com/mitre/serverless-inspec
@aaronlippold Are you requesting something different than the function that is defined here and is called here? I have it set up now so that you can tag an AWS Config Rule with Info, Low, Medium, High, or Critical. Should I add the ability to assign a numerical value between 0 and 1 in addition to this?

Also, totally happy to collaborate on lambda work.

@rx294 @rbclark @ejaronne and @aaronlippold meet earlier today and @rx294 will be providing some feedback on our suggested direction for the PR.

Great start so far, but we have a few thoughts on some backend learning we have done but not yet put forward on a docs page about 'how to develop mappers', 'what and how you should transform data', 'do and don't on data transform assumptions', 'what to embed, what to hardcode, and what to reference' etc. all things we should have documented.

The list above I think would be great sections on a 'contributing_a_mapper.md' which may come out of our collaboration on this PR.

Looking forward to it.

aaronlippold · 2021-02-17T03:11:38Z

@jkufro some additions to the unit and functional tests as well with some representative source and expected converted data would be great as well so we keep the code coverage up to par.

jkufro · 2021-02-17T14:44:11Z

@jkufro
The current design of the aws_config_mapper requires user setup (ie tag placement) on the aws_config rules, which will be a deviation from pattern used in other mappers. Please see below suggestions to align the aws_config_mapper to the general pattern in used in heimdall_tools.

This will help with reusability and consistency of the HDFs generated by aws_config_mapper across multiple users.

Control Title mapped title of the AWS_Config rule (as is the case currently)

Description: Parsed from the aws_config rule description feild.

Check Text to include the config rule arn and parameters used for the rule. Consider formatting the data fields with linefeeds.

Set the severity to the "average" of 0.5 (medium)" for all controls.

Nist Tag: Create a CSV file mapping config_rule titles to NIST 800-53 mappings as in examples here https://github.com/mitre/heimdall_tools/blob/master/lib/data/owasp-nist-mapping.csv and use it in mapping NIST tags to the controls. This way mapping to NIST controls stays consistant and centralized at heimdall_tools codebase.

Remove references to cce_id cis_impact cis_rid fix rationale cweid confidence cis_level at this iteration.

Please add sample hdf jsons to this location https://github.com/mitre/heimdall_tools/tree/master/sample_jsons

Please add readme updates

Please resolve merge conflicts.

@rx294 One major aspect of our use case is continued development/implementation of custom AWS Config rules AWS Docs. My concern with hardcoding Config Rule Name -> 800-53 Control mappings inside the heimdall_tools repository is that as we (or others) develop new rules, we would be required to update the heimdall_tools code to perform proper mappings. Another issue with adding custom rule names to a heimdall_tools CSV mapper is that the source code for each custom rule may not be available open source, so the heimdall_tools CSV for AWS Config could become bloated with rules that only apply to a handful of projects. The original reason I chose to avoid a CSV when creating this mapper was to allow the simplicity of creating a rule with the proper tags to be the only thing necessary to fully configure a rule for mapping & use within heimdall.

EDIT: Here are some examples of Python custom rules made by AWS: https://github.com/awslabs/aws-config-rules/tree/master/python

lib/heimdall_tools/aws_config_mapper.rb

jkufro · 2021-02-19T19:32:35Z

@aaronlippold @rbclark @rx294 Please review the latest updates that address the requested changed. Thanks!

lib/heimdall_tools/aws_config_mapper.rb

README.md

ejaronne

I have several comments. This is the first time we've introduced an idea of end-user defined 800-53 control assertions, which may cause confusion or complaints against this tool. Most assume the mapping comes from either the original approved standard or a mapping set in the heimdall-tools repo. The heimdall-tools repo owner can own-up to these, but cannot to user-defined mappings. Note: CAAT file export does not carry over the "(user provided)" text.

lib/data/aws-config-mapping.csv

sample_jsons/aws_mapper/aws_config_hdf.json

lib/heimdall_tools/help/aws_config_mapper.md

ejaronne · 2021-02-26T02:50:09Z

lib/heimdall_tools/help/aws_config_mapper.md

+        region = us-gov-west-1
+        ```
+
+Additional Rule Mappings:


I would rephrase this "feature" to "Custom-rule NIST SP 800-53 Security Control mappings" and restate the paragraph as:
"For user-defined custom AWS config rules (NOT the default rules provided by AWS), users may provide their own assertion as to the NIST SP 800-53 Security Controls touched on by their user-defined custom AWS config rules. This may be provided..."

Also, this is not "additional mapping". it is "--custom-mapping"

pushed update that changes the description and keyword arg name

For the first release, may we please not reveal in the cli help the custom-mapping feature until we've had a chance to adapt Heimdall and its CAAT export to clearly expose the (user-provided) aspect?

disabled cli arg for the custom mapping for the time being

rx294 · 2021-02-26T19:53:00Z

lib/heimdall_tools/cli.rb

+    option :output, required: true, aliases: '-o'
+    option :verbose, type: :boolean, aliases: '-V'
+    def aws_config_mapper
+      hdf = HeimdallTools::AwsConfigMapper.new(options[:addl_mapping]).to_hdf


@jkufro I think you intended it to be options[: custom_mapping] and not options[:addl_mapping]

rx294 · 2021-02-26T19:59:27Z

lib/heimdall_tools/aws_config_mapper.rb

+    def initialize(addl_mapping, verbose = false)
+      @verbose = verbose
+      @default_mapping = get_rule_mapping(AWS_CONFIG_MAPPING_FILE)
+      @addl_mapping = addl_mapping.nil? ? {} : get_rule_mapping(addl_mapping)


Please unify to custom_mapping in sync with usage in the cli.rb.

rx294 · 2021-02-26T20:01:50Z

lib/heimdall_tools/aws_config_mapper.rb

+    # If there is overlap in rule names from @default_mapping and @addl_mapping, 
+    # then the tags from both will be added to the rule.
+    def to_hdf
+      controls = @issues.filter_map do |issue|


@jkufro filter_map is only available in ruby 7.x which will limit heimdall_tools to ruby 7.x.

Please use and alternate implementation that doesn't not force a > 2.7 dependency.

Changing back to map. I had changed this to filter_map previously for something that I have since removed.

rx294 · 2021-02-26T20:10:26Z

README.md

@@ -197,6 +198,20 @@ FLAGS:
 example: heimdall_tools jfrog_xray_mapper -j xray_results.json -o xray_results_hdf.json
 ```

+## aws_config_mapper


Please include details from your excellent documentation from aws_config_mapper.md here in the Readme as well.

Signed-off-by: Justin Kufro <[email protected]>

ejaronne

Looks good, Thanks!

rx294 · 2021-03-01T15:45:00Z

LGTM, thanks @jkufro . Please resolve conflicts, that should be the last fix.

aaronlippold · 2021-03-01T16:43:31Z

Just a reminder, please add an issue to the board to revisit the issue of 'user provided' data - aka overlay data - in our workflow so we can make sure to address the capability we didn't want to try to push in v1.0

rx294 · 2021-03-01T16:53:55Z

Just a reminder, please add an issue to the board to revisit the issue of 'user provided' data - aka overlay data - in our workflow so we can make sure to address the capability we didn't want to try to push in v1.0

https://github.com/mitre/heimdall_tools/issues/73

aaronlippold · 2021-03-01T16:57:19Z

👍🏻 great job all

rbclark reviewed Feb 10, 2021

View reviewed changes

lib/heimdall_tools/aws_config_mapper.rb Outdated Show resolved Hide resolved

rbclark reviewed Feb 10, 2021

View reviewed changes

lib/heimdall_tools/help/aws_config_mapper.md Show resolved Hide resolved

jkufro force-pushed the awsConfig branch from 93cc533 to 7ab1f62 Compare February 10, 2021 15:15

rbclark reviewed Feb 10, 2021

View reviewed changes

jkufro force-pushed the awsConfig branch 2 times, most recently from f328805 to d89d44e Compare February 10, 2021 15:28

rbclark requested a review from rx294 February 10, 2021 15:30

jkufro force-pushed the awsConfig branch 2 times, most recently from b9c357d to def5397 Compare February 10, 2021 16:05

aaronlippold requested a review from ejaronne February 16, 2021 03:14

aaronlippold added the enhancement New feature or request label Feb 16, 2021

aaronlippold assigned rbclark, rx294 and jkufro Feb 16, 2021

aaronlippold requested a review from yarick February 16, 2021 03:43

rbclark removed their assignment Feb 16, 2021

jkufro force-pushed the awsConfig branch 3 times, most recently from a180198 to fc0c2c3 Compare February 17, 2021 19:37

jkufro commented Feb 17, 2021

View reviewed changes

lib/heimdall_tools/aws_config_mapper.rb Outdated Show resolved Hide resolved

jkufro force-pushed the awsConfig branch from fc0c2c3 to 5e4fd26 Compare February 18, 2021 18:05

aaronlippold reviewed Feb 22, 2021

View reviewed changes

lib/heimdall_tools/aws_config_mapper.rb Outdated Show resolved Hide resolved

jkufro commented Feb 25, 2021

View reviewed changes

README.md Show resolved Hide resolved

jkufro force-pushed the awsConfig branch 2 times, most recently from 6133b0f to 4b11835 Compare February 25, 2021 17:35

ejaronne requested changes Feb 26, 2021

View reviewed changes

jkufro force-pushed the awsConfig branch from 4b11835 to 6a3f140 Compare February 26, 2021 14:50

jkufro requested a review from ejaronne February 26, 2021 14:51

rx294 reviewed Feb 26, 2021

View reviewed changes

Add mapper for AWS Config Rules

9075325

Signed-off-by: Justin Kufro <[email protected]>

jkufro force-pushed the awsConfig branch from 6a3f140 to 9075325 Compare March 1, 2021 13:49

jkufro requested a review from rx294 March 1, 2021 13:53

ejaronne approved these changes Mar 1, 2021

View reviewed changes

Merge branch 'master' into awsConfig

ec7cf15

rx294 merged commit 1ddece6 into mitre:master Mar 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add mapper for AWS Config Rules #68

Add mapper for AWS Config Rules #68

jkufro commented Feb 10, 2021 •

edited

Loading

rbclark Feb 10, 2021

jkufro Feb 10, 2021

rbclark Feb 10, 2021

jkufro Feb 10, 2021 •

edited

Loading

ejaronne Feb 16, 2021 •

edited

Loading

ejaronne Feb 16, 2021

rbclark Feb 10, 2021

aaronlippold commented Feb 16, 2021

jkufro commented Feb 16, 2021

rbclark commented Feb 16, 2021

rx294 commented Feb 16, 2021 •

edited

Loading

aaronlippold commented Feb 17, 2021

aaronlippold commented Feb 17, 2021

jkufro commented Feb 17, 2021 •

edited

Loading

jkufro commented Feb 19, 2021

ejaronne left a comment

ejaronne Feb 26, 2021

ejaronne Feb 26, 2021

jkufro Feb 26, 2021

ejaronne Feb 27, 2021

jkufro Mar 1, 2021

rx294 Feb 26, 2021

jkufro Mar 1, 2021

rx294 Feb 26, 2021 •

edited

Loading

jkufro Mar 1, 2021

rx294 Feb 26, 2021

jkufro Mar 1, 2021

rx294 Feb 26, 2021

jkufro Mar 1, 2021

ejaronne left a comment

rx294 commented Mar 1, 2021 •

edited

Loading

aaronlippold commented Mar 1, 2021

rx294 commented Mar 1, 2021

aaronlippold commented Mar 1, 2021

Add mapper for AWS Config Rules #68

Add mapper for AWS Config Rules #68

Conversation

jkufro commented Feb 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkufro Feb 10, 2021 • edited Loading

Choose a reason for hiding this comment

ejaronne Feb 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronlippold commented Feb 16, 2021

jkufro commented Feb 16, 2021

rbclark commented Feb 16, 2021

rx294 commented Feb 16, 2021 • edited Loading

aaronlippold commented Feb 17, 2021

aaronlippold commented Feb 17, 2021

jkufro commented Feb 17, 2021 • edited Loading

jkufro commented Feb 19, 2021

ejaronne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rx294 Feb 26, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ejaronne left a comment

Choose a reason for hiding this comment

rx294 commented Mar 1, 2021 • edited Loading

aaronlippold commented Mar 1, 2021

rx294 commented Mar 1, 2021

aaronlippold commented Mar 1, 2021

jkufro commented Feb 10, 2021 •

edited

Loading

jkufro Feb 10, 2021 •

edited

Loading

ejaronne Feb 16, 2021 •

edited

Loading

rx294 commented Feb 16, 2021 •

edited

Loading

jkufro commented Feb 17, 2021 •

edited

Loading

rx294 Feb 26, 2021 •

edited

Loading

rx294 commented Mar 1, 2021 •

edited

Loading