-
Notifications
You must be signed in to change notification settings - Fork 682
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow inspec check
to ignore only_if
#2250
Conversation
@@ -133,7 +134,16 @@ def to_s | |||
|
|||
define_method :only_if do |&block| | |||
return unless block | |||
return if @skip_file == true || block.yield == true | |||
return if @skip_file == true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we just set @skip_file
to false if we are in @backend.mock_transport?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we can because Unit tests also use a mock backend. Doing so would make it impossible to test only_if
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just two quick clean-up items and this looks good.
I wonder if there's a better way to do this without having to pass around data as to whether we're mocking or not. We already tightly-couple ourselves to the mock backend for unit tests, so I'm afraid of future edge cases.
Would it make sense to introduce the concept of an "operating mode" class var we can set and query? For example, in the Thor method when inspec check
is run:
InSpec::OperatingMode.set_mode(:check)
... and we can flat-out skip only_if evaluation with something like:
return if InSpec::OperatingMode.is_check?
# or
return if InSpec::OperatingMode.current_mode == :check
@jerryaldrichiii @arlimus @chris-rock -- thoughts?
lib/inspec/control_eval_context.rb
Outdated
# Ignore error if a mock connection | ||
# Example: `inspec check` with `only_if { os.name.include?('windows' }` | ||
return if @backend.mock_transport? | ||
raise e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can eliminate the e
here - raise
within a rescue
will re-raise the original exception.
lib/inspec/rule.rb
Outdated
@__skip_rule ||= !yield | ||
rescue => e | ||
return if @__mock == true | ||
raise e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can eliminate the e
here - raise
within a rescue
will re-raise the original exception.
Thanks @adamleff didn't know that about In regards to |
The more I think about it the more I am not sure we are solving the right thing here. We have two problems that I see here:
The first issue can also happen outside of the For the second one. We could try to catch any exceptions for |
@chris-rock I think the That's why I think we solve this with exposing a way for InSpec to communicate that we're in |
Hey @jerryaldrichiii! @chris-rock and I met today to discuss this change. First off, we agree that we should probably add a PR to Train to correctly build an Secondly, we'd like to take a different approach with this PR. Essentially, rather than tightly coupling ourselves to the mock backend, we'd like to pass through a flag that indicates we're in "check mode" and then, if so, tell the
Let me know if you have any questions. |
7130ca1
to
497a0de
Compare
lib/inspec/control_eval_context.rb
Outdated
@@ -53,6 +53,7 @@ def initialize(backend, conf, dependencies, require_loader) | |||
@dependencies = dependencies | |||
@require_loader = require_loader | |||
@skip_file = false | |||
@skip_only_if_eval = conf['check_mode'] || false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to draw a line at what things care about the concept of "check mode". In my opinion, the line needs to be drawn at the profile. Controls (and their evaluation contexts) should not care about the mode in which InSpec was executed but rather whether they should care about only_if or not.
Can we find a way to gracefully pass a bool for skip_only_if_eval from the profile to the control instead of passing check_mode here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are your thoughts about just passing it all the way from the CLI, e.g. o[:skip_only_if_eval] = true
vs o[:check_mode] = true
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While you can claim YAGNI, what if something else in the future needs to care about whether we're in check_mode or not? And should the CLI really be directing how a control is evaluated?
The CLI knows it's in check mode, the profile needs to know it's in check mode (because it's the thing being checked), and then the profile should use that information to determine whether or not certain activities should take place. I still believe the line I drew and described above is the right separation of duties and logic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, I moved the line up to Profile and then passed skip_only_if_eval
it to the ControlEvalContext
c2492c4
to
2d5b12e
Compare
When using `inspec check` a mock Train backend is created. This means that the following would raise an error because `os.name` is `nil` ``` only_if { os.name.include?('anything') } ``` Since `inspec check` isn't concerned with the evaluation of `only_if` this skips those checks if the block given raises an error. Signed-off-by: Jerry Aldrich <[email protected]>
Signed-off-by: Jerry Aldrich <[email protected]>
Signed-off-by: Jerry Aldrich <[email protected]>
Signed-off-by: Jerry Aldrich <[email protected]>
2d5b12e
to
e39e11e
Compare
Signed-off-by: Jerry Aldrich <[email protected]>
@@ -96,6 +96,7 @@ def initialize(source_reader, options = {}) | |||
@attr_values = options[:attributes] | |||
@tests_collected = false | |||
@libraries_loaded = false | |||
@check_mode = options[:check_mode] || false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Little Ruby nugget - careful with ||
and default booleans. For example, if you wanted the default to be true
and you wrote this like:
@check_mode = options[:check_mode] || true
... it could never be false! If the user passed in false
as the value in options[:check_mode]
, it would fail the left side of the ||
and then right side would evaluate.
I tend to use #fetch
on Hash
for this instead, just in case:
@check_mode = options.fetch(:check_mode, false)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, thanks for the tip. Never thought of that.
@@ -31,7 +31,7 @@ def foobar | |||
let(:eval_context) do | |||
c = Inspec::ControlEvalContext.create(profile_context, resource_dsl) | |||
# A lot of mocking here :( | |||
c.new(backend, mock(), mock(), mock()) | |||
c.new(backend, {}, mock(), mock(), false) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add a comment here that enumerates the options ControlEvalContext expects so the next reader doesn't have to hop back to understand? I think that would help greatly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great idea, I pushed a commit that list the options that are mocked. Let me know if I need to modify the wording.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One request-for-comment, but otherwise, this is good to go.
Signed-off-by: Jerry Aldrich <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic fix, thank you @jerryaldrichiii !!
This fixes #2175
This fixes #2237
When using
inspec check
a mock Train backend is created. This means that the following would raise an error becauseos.name
isnil
Since
inspec check
isn't concerned with the evaluation ofonly_if
this skips those checks if the block given raises an error.