Piped input does not respect excluded paths

[BenjaminEngeset]

Description of the issue

PSRule does not respect exluded paths in ps-rule.yaml

To Reproduce

input:
  pathIgnore:
    # Ignore common files that don't need analysis.
    - "**/bicepconfig.json"
    - "*.md"
    - "*.png"
    - ".azuredevops/"

    # Exclude Bicep module files.
    - "modules/**/*.bicep"

    # Exclude JSON module files.
    - "modules/**/*.json"

    # Include test/main.test.bicep files from modules.
    - "!modules/**/test/*.bicep"

I also have checked ps-rule branch in ALZ-Bicep and I can see it's solved the same way there?

Also tried

- "!/modules/**/*.test.bicep"

The task

Install-Module PSRule.Rules.Azure -Scope CurrentUser -Force;
git diff --diff-filter=d HEAD^ HEAD --name-only | ForEach-Object { Get-ChildItem $_ } | Assert-PSRule -Format File -Module 'PSRule.Rules.Azure' -Baseline 'Azure.GA_2022_03' -OutputFormat NUnit3 -OutputPath 'reports/ps-rule-results.xml'

Expected behaviour

Should only traverse test folder with it's underlaying .bicep content.

Error output

2022-06-17T11:43:47.8240390Z ##[error]Failed to expand bicep source '/home/vsts/work/1/s/modules/authorization/roleassignments-automation-account/main.bicep'. Exception calling "GetBicepResources" with "3" argument(s): "Unable to expand resources because the source file '/home/vsts/work/1/s/modules/authorization/roleassignments-automation-account/main.bicep' was not valid. An error occurred evaluating expression '[length(parameters('parPrincipalIds'))]' line 92. The parameter named 'parPrincipalIds' was not set or a defaultValue was defined."
2022-06-17T11:43:50.8662218Z 
2022-06-17T11:43:50.8664516Z  -> /home/vsts/work/1/s/modules/authorization/roleassignments-automation-account/test/main.test.bicep : Microsoft.Resources/deployments [2/2]
2022-06-17T11:43:50.8668930Z 
2022-06-17T11:43:50.8675905Z     [PASS] Azure.Template.ParameterStrongType (AZR-000227)
2022-06-17T11:43:50.8679748Z     [PASS] Azure.Template.ExpressionLength (AZR-000228)
2022-06-17T11:43:50.8905808Z 
2022-06-17T11:43:50.8908502Z  -> /home/vsts/work/1/s/modules/authorization/roleassignments-automation-account/test/main.test.bicep : Microsoft.Resources/deployments [2/2]
2022-06-17T11:43:50.8909925Z 
2022-06-17T11:43:50.8911413Z     [PASS] Azure.Template.ParameterStrongType (AZR-000227)
2022-06-17T11:43:50.8912198Z     [PASS] Azure.Template.ExpressionLength (AZR-000228)

As seen it's also trying to traverse thru main.bicep which is not buildable.

Module in use and version:

• Module: PSRule
• Version: v2.1.0

I am quite unsure if this is a bug or just human error from my side.

[BernieWhite]

@bengeset96 Thanks for reporting the issue. Yes, input.pathIgnore was never really intended to filter passed in objects, because you could pre-filter these objects in PowerShell before pipeline the objects to PSRule.

You could do this via Get-ChildItem $_ -Exclude 'bicepconfig.json'.

That said I can see why you would want to do this, and the support within PSRule makes it a desirable option. Although it could be confusing to have it enabled by default.

[BenjaminEngeset]

@BernieWhite

Thanks for the great explanation, it all makes sense now!

In this particular case, PSRule is used for validation on PR in Bicep module repo. Each module folder has main.bicep and test/main.test.bicep which is the deployable one.

Then it seems like it should be like this based on your explanation?

Install-Module PSRule.Rules.Azure -Scope CurrentUser -Force;
git diff --diff-filter=d HEAD^ HEAD --name-only | ForEach-Object { Get-ChildItem $_ -Exclude 'main.bicep' } | Assert-PSRule -Format File -Module 'PSRule.Rules.Azure' -Baseline 'Azure.GA_2022_03' -OutputFormat NUnit3 -OutputPath 'reports/ps-rule-results.xml'

[BernieWhite]

@bengeset96 Yes, that would be correct as a workaround.

[BenjaminEngeset]

@BernieWhite

Thank you, awesome stuff!

I'm sucessfully using -Exclude for now as I were not able to make it work with pre-release version for PSRule

Install-Module PSRule.Rules.Azure -AllowPrerelease -Scope CurrentUser -Force;

Looking forward to official release when it arrives.

[BernieWhite]

@bengeset96 No worries.

The release notes may not be clear. I'll update that.

You can try the pre-release by setting the Input.IgnoreObjectSource option to true.

[BenjaminEngeset]

@BernieWhite

I am. Checked your PR.

input:
  pathIgnore:
    # Ignore common files that don't need analysis.
    - "**/bicepconfig.json"
    - "*.md"
    - "*.png"
    - ".azuredevops/"

    # Exclude Bicep module files.
    - "modules/**/*.bicep"

    # Exclude JSON module files.
    - "modules/**/*.json"

    # Include test/main.test files from modules.
    - "!modules/**/test/*.bicep"

  ignoreObjectSource: true

With Install-Module PSRule.Rules.Azure -AllowPrerelease -Scope CurrentUser -Force;

Does not work. I think it's because of -AllowPrerelease is not working the way I have configured it. Have not tested with the ADO task which has this input available.

2022-06-20T11:05:47.1131091Z �[91mNew-PSRuleOption: �[91mException calling "FromFileOrEmpty" with "2" argument(s): "(Line: 16, Col: 3,
2022-06-20T11:05:47.1132864Z Idx: 337) - (Line: 16, Col: 3, Idx: 337): While parsing a block mapping, did
2022-06-20T11:05:47.1133572Z not find expected key."
2022-06-20T11:05:47.1134276Z �[0m
2022-06-20T11:05:47.2011636Z ##[error]PowerShell exited with code '1'.

[BernieWhite]

@bengeset96 Generally, PSRule doesn't care about additional keys. So adding ignoreObjectSource shouldn't break anything even if PSRule doesn't understand it.

Without seeing your full config i'd assume that you have a : or an extra space in the wrong place somewhere around line 16 on ps-rule.yaml.

[BenjaminEngeset]

@BernieWhite

It was all on me, you were absolutely correct.

Seems like it's a bit tricky to use pre-release version.

Tried to use these modules, but still it's using 2.1.0.

Install-Module -Name PowerShellGet -Repository PSGallery -Scope CurrentUser -Force
Install-Module PSRule.Rules.Azure -AllowPrerelease -Scope CurrentUser -Force;

https://microsoft.github.io/PSRule/v2/install-instructions/#pre-release-versions

Is there another way to force the usage of pre-release version as you know of?

Also added

requires:
  PSRule: "@pre >=2.2.0-B50052"

[BernieWhite]

@bengeset96 Currently requires is just a constraint that can be set, more so that a consistent version of PSRule is used across a team. It errors out when the constraint is not met, but doesn't install the specific version. But we like to implement that feature at some point.

To install a pre-release version of the PSRule engine just install PSRule directly, instead of relying on the dependency chain from PSRule for Azure. Try:

Install-Module -Name PowerShellGet -Repository PSGallery -Scope CurrentUser -Force;
Install-Module PSRule -AllowPrerelease -Scope CurrentUser -Force;
Install-Module PSRule.Rules.Azure -AllowPrerelease -Scope CurrentUser -Force;

[BenjaminEngeset]

@BernieWhite I am aware, I do really like the requires as it ensures that teams are using the correct version in a controlled manner. Both for official and pre-release versions (when the pre-release has wanted functionality like 2.2.0-B50052 so I want to lock it for that spesific version for an instance).

Then I understand, so you actually have to install it directly, I taught it would work with the dependency chain.

With your suggestion it works perfectly fine and once again you have made a superb contribution that I am truly grateful for!

[BernieWhite]

@bengeset96 No worries. And thanks for the feedback.