Cannot compile blueprints which have variables as values

[ricmestre]

@FabienTschanz On your NoEscape PR you started escaping dollar signs on strings, any reason behind this? I generate blueprints dynamically and use variables throughout them but now I cannot compile them to MOF since they incorrectly have escaped values which are supposed to be variables, see below.

EXOAcceptedDomain "EXOAcceptedDomain-$OrganizationName"
{
    ApplicationId         = `$EXOApplicationId;
    CertificateThumbprint = `$EXOCertThumbprint;
    DomainType            = "Authoritative";
    Ensure                = "Present";
    Identity              = "`$OrganizationName";
    MatchSubDomains       = $False;
    OutboundOnly          = $True;
    TenantId              = `$OrganizationName;
}

[ricmestre]

Well, that's not the only problem now it seems, it also botched Convert-DSCStringParamToVariable, I have big CIM instances like the below where now the double quotes are removed which is not correct, see below the Value property, in the XML after the equal signs the values should be enclosed with double quotes and now they are being removed automatically, ?xml version=1.0 ? should be ?xml version="1.0" ?

            OmaSettings           = @(
                MSFT_MicrosoftGraphomaSetting{
                    Description = 'Chrome ADMX Ingestion'
                    DisplayName = 'Chrome'
                    IsEncrypted = $False
                    OmaUri = './Device/Vendor/MSFT/Policy/ConfigOperations/ADMXInstall/Chrome/Policy/ChromeAdmx'
                    Value = '<?xml version=1.0 ?>
<policyDefinitions revision=1.0 schemaVersion=1.0>
  <!--chrome version: 92.0.4515.131-->
  <policyNamespaces>
    <target namespace=Google.Policies.Chrome prefix=chrome/>
    <using namespace=Google.Policies prefix=Google/>
    <using namespace=Microsoft.Policies.Windows prefix=windows/>
  </policyNamespaces>
  <resources minRequiredRevision=1.0/>
  <supportedOn>
[...]

EDIT: I managed to patch an old M365DSC version and change my code to use Get-M365DSCExportContentForResource with NoEscape which solves this problem, but now there's another one where the authentication methods are hardcoded with $ConfigurationData.NonNodeData.* but what I want is to use my own variables such as $EXOApplicationId or $EXOCertThumbprint. I know where the culprit is, but I want to know what's the reason to hardcode these values now. Mind that the dollar sign escaping is a separate issue, to fix it is just a matter of removing a couple of replaces and it fixes the issue but also here I don't know the reason why you added those replaces.

[FabienTschanz]

@FabienTschanz On your NoEscape PR you started escaping dollar signs on strings, any reason behind this? I generate blueprints dynamically and use variables throughout them but now I cannot compile them to MOF since they incorrectly have escaped values which are supposed to be variables, see below.

EXOAcceptedDomain "EXOAcceptedDomain-$OrganizationName"
{
ApplicationId = $EXOApplicationId; CertificateThumbprint = $EXOCertThumbprint;
DomainType = "Authoritative";
Ensure = "Present";
Identity = "$OrganizationName"; MatchSubDomains = $False; OutboundOnly = $True; TenantId = $OrganizationName;
}

1. Escaping dollar signs inside of strings was a wish to support "special" characters like the dollar sign. Otherwise, they would be interpreted as variables (but they wouldn't have any content) and report as a drift. I once had this issue as well.

2. The escaping of the dollar sign on ApplicationId etc. is an issue I'm just working on fixing. This comes from changes in Microsoft365DSC.

3. For the authentication methods, I'm not aware of any real changes there. What I did was to move the Update-M365DSCExportAuthenticationResults from inside the resource to Get-M365DSCExportContentForResource. I'm not aware of any other changes that would step in here.

4. Not quite sure why the quotes are missing from the Value property, but with my patched version (from point 2) it will export them properly.

[FabienTschanz]

PR is open: microsoft/Microsoft365DSC#5767

Additionally, Convert-DSCStringParamToVariable was afaik not touched by me. The behavior of it should still be the same. I did a full Intune workload export of my tenant with the latest changes and it all worked without issues, so I'm not quite sure how some of your issues exist. Maybe you can take a look at the changes in the PR? Sorry about that, wasn't my intention at all.

[ricmestre]

The problem with the double quotes I fixed it by using Get-M365DSCExportContentForResource ... -NoEscape $NoEscape

But that PR doesn't fix the other 2 issues, first regarding the authentication methods if you call Update-M365DSCExportAuthenticationResults then it hardcodes those values, I don't want that, I want to use my own variables or whatever string I might want.

Second one about the $ escaping, what about the fact that first level strings are always enclosed in double quotes but inside CIM instances they are actually enclosed in single quotes? With your solution you'll end up with "`$" and '`$' which actually outputs $ and `$ respectively. At the very least you need to remove the escaping from CIM instances since it doesn't make sense, and I understand now your use-case but what if I really want to spit a variable into the DSCBlock and not an escaped version of it, see below, with this latest release now you're forcing me to always have the escaped version and I won't know which ones should be variables and which ones should be literal strings.

ExceptIfSentToMemberOf    = @("LegalTeam@$OrganizationName");

[FabienTschanz]

Can you walk me through how you create the blueprints so that I can accurately depict what you're doing? Sorry if I don't understand it yet, I haven't been using blueprints at all, so would really appreciate if you could help me guide you to your problem. Then I can take a closer look and check what's going on.

For the $ escaping, is this also blueprint related? If yes, does it help if you add the property name which contains a variable to the -NoEscape parameter?

[ricmestre]

I have markdown files of the resources which are the same ones included in M365DSC docs/docs folder, then I append a Value column to all tables and then insert the values of the properties there, so imagine in that case that you have a row where ParameterName is ExceptIfSentToMemberOf and Value is LegalTeam@$OrganizationName, when I convert that markdown into the blueprint then that line ends up as ExceptIfSentToMemberOf = @("LegalTeam@$OrganizationName"); because that parameter is a string array, in case I want it to be escaped I can also do that by adding the ` character myself, with this release I can't do that because it's always forced.

And no, I cannot do that because it's up to the admins using my solution where to use these variables so I don't know the names of the properties I might need escape, albeit in this example I actually know because it was me who actually did it BUT as part of a big hammer I have in my code which replaces all notions of the fqdn of the tenant, its short name as well or even the GUID from the original retrieved blueprint and then I insert the replaced strings into the markdown files, when I do the reverse process then it's where I'm encountering now the issue.

[FabienTschanz]

Now I understand why you're affected by the Update-M365DSCExportAuthenticationResults move... You're directly using the Get-M365DSCExportContentForResource cmdlet, right? And everything that's set as an authentication property (e.g. ApplicationId or CertificateThumbprint) will of course be replaced... Previously this was done in the resource, but because each and every resource had this call, I moved it to simplify maintenance of it. But now this pops up, argh.

What do you think of a switch parameter that would let you skip the replacement of the authentication properties? E.g. -SkipUpdateAuthenticationResults? This would probably solve your issue with the replacement of the auth properties.

[ricmestre]

Yes I'm using Get-M365DSCExportContentForResource directly and sure that fix would be enough, and you could also use the same same type of fix for the $ escaping. Let's cater for the big audience who don't use any of these things by themselves anway so make the default as you have right now but add those switches that I can use and then I can adapt my code to use them.

I already spoke a few times with Nik about this but essentially whenever there's any change to DSCParser or ReverseDSC even if it's only a very small change I'm almost always affected and need to adapt that change to work with what I have, other times I need to also adapt my code which is the case for these 2 issues about the auth methods and the $ escaping. I usually test the PRs on these modules because I already know they'll cause me pain, but I'm extremely busy trying to make things work with old versions and yesterday I had a few minutes to try the new ReverseDSC and right away saw several thousands of errors on my unit tests, not funny 😆