Running M365 DSC commands in CI pipeline with firewall blocking access to PowerShell Gallery

[RosalindHook]

I am attempting to run Microsoft 365 DSC (M365 DSC) commands within a CI pipeline in GitLab. However, my organization has blocked access to the PowerShell Gallery due to network security policies, and I am unable to set up a proxy to access it.

As a workaround, I downloaded the required DSC modules and pushed them to Artifactory (a remote repository). In the pipeline, I am downloading and unzipping the modules, and then running the following command to import the module:

Import-Module "$modulePath/Microsoft365DSC.psd1"

After successfully importing the module, I attempt to execute the Export-M365DSCConfiguration command using secret credentials. However, I encounter the following error:

Write-Error: Failed to export M365 DSC Configuration: The term 'Connect-M365Tenant' is not recognized as the name of a cmdlet, function, script file, or executable program.

When running the script locally, I would typically use the Update-M365DSCDependencies command to ensure all required dependencies are installed and updated, but this is not possible in my current CI pipeline setup due to the lack of PowerShell Gallery access. I understand that M365DSC relies on several other modules for its functionality, which are likely out of date in my current environment.

Questions/Clarifications:

1. Is there a way to manually identify which specific modules M365DSC depends on so that I can download and update them independently?
2. Is it necessary to have the most up-to-date version of M365DSC and all its dependencies to resolve the Connect-M365Tenant error, or is there another potential solution I can implement in my CI pipeline to bypass this issue?

[FabienTschanz]

@RosalindHook The required modules with their versions for the current version of Microsoft365DSC can be found here: Modules/Microsoft365DSC/Dependencies/Manifest.psd1. If you install the modules in the system context before running any other commands of the Microsoft365DSC module, you should be fine.

[RosalindHook]

Thank you @FabienTschanz this is really helpful.

I am having a related issue with the dependency PSDesiredStateConfiguration. If I run the command to update all dependencies, everything updates to the versions in the manifest with the exception of PnP.PowerShell. I get this warning:

WARNING: The dependency {PSDesiredStateConfiguration} requires PowerShell Core. Please run Update-M365DSCDependencies
in PowerShell Core.

But I thought the recommendation was to use PowerShell 5.x with DSC? Therefore what should I do about this one dependency requiring a different version?

[FabienTschanz]

If you were to use PowerShell 7, you would need to install that module in the PowerShell 7 shell. If you don't use that, you can safely ignore the message. Current suggestion is, as you wrote, PS 5.1.

[RosalindHook]

Was about to close this issue but actually I have one more related question. Given that the recommendation is to use PowerShell 5.1, how should I download this for use in a CI pipeline... looking at the Microsoft PowerShell installation instructions, all these versions appear to be 7+ here

[FabienTschanz]

The installation instructions are for the "new" PowerShell, meaning PowerShell 7. If you want to use Microsoft365DSC in a CI pipeline, you need to register a Windows machine with PowerShell 5.1. as your CI/CD pipeline runner and schedule the jobs to be run on that machine. You can find a whitepaper for GitHub on how it's done here: Microsoft365DSC Whitepaper

[RosalindHook]

Thank you....
If I run the command 'Update-M365DSCDependencies' in PowerShell7 and the modules are higher versions than those set out in the manifest, is that a problem?
An example I just found is the manifest says DSCParser should be 2.0.0.14 but running the update dependencies command gives me 2.0.0.15 - there are a few others too where there is a discrepancy (always seems to be higher versions than in the manifest)

[RosalindHook]

PnP.PowerShell is another. Manifest statesr 1.12.0 but updating dependencies gives me 2.12.0
And I am particularly concerned about this one as I think it's conflicting with Microsoft.Graph

[FabienTschanz]

@RosalindHook The intended way to use Microsoft365DSC is to install the main module in PS 5.1, run Update-M365DSCModule, and if you then switch to PS7, you run the command again. After that, you'll be up to date with everything.

• If the modules are higher than in the manifest, they'll be downgraded to the same version as in the manifest

There shouldn't be any discrepancy between what Update-M365DSCDependencies does and what is written in the manifest. Can you give an example what you mean by that? Also, I don't want any misunderstandings to come up - Microsoft365DSC is not yet able to fully run with PS7. Only the export capability is able to run there. Start-DscConfiguration still requires PS5.1 and does not work otherwise.

[RosalindHook]

Thank you - at the moment I am just trying to use the export capability (which I have succeeded in doing for some workloads locally, but I am now trying to automate and move to a pipeline), so that works for my purposes for the time being.

On the discrepancy point, I ran the cmdlet 'Update-M365DSCModule' in Powershell 5. Below is a full list of the modules that installed as a result of this command. Many of these are slightly higher versions than in the manifest here by the looks of things? (e.g. all these Microsoft.Graph modules are version 2.26.1 whereas the manifest states version 2.25.0, ReverseDSC is 2.0.0.27 but the manifest states 2.0.0.22)

Installed versions:
Installing ExchangeOnlineManagement version {3.7.1}
Installing Microsoft.Graph.Applications version {2.26.1}
Installing Microsoft.Graph.Beta.Applications version {2.26.1}
Installing Microsoft.Graph.Beta.DeviceManagement version {2.26.1}
Installing Microsoft.Graph.Beta.Devices.CorporateManagement version {2.26.1}
Installing Microsoft.Graph.Beta.DeviceManagement.Administration version {2.26.1}
Installing Microsoft.Graph.Beta.DeviceManagement.Enrollment version {2.26.1}
Installing Microsoft.Graph.Beta.NetworkAccess version {2.26.1}
Installing Microsoft.Graph.Beta.Identity.DirectoryManagement version {2.26.1}
Installing Microsoft.Graph.Beta.Identity.Governance version {2.26.1}
Installing Microsoft.Graph.Beta.Identity.SignIns version {2.26.1}
Installing Microsoft.Graph.Beta.Reports version {2.26.1}
Installing Microsoft.Graph.Beta.Search version {2.26.1}
Installing Microsoft.Graph.Beta.Teams version {2.26.1}
Installing Microsoft.Graph.DeviceManagement.Administration version {2.26.1}
Installing Microsoft.Graph.Beta.DirectoryObjects version {2.26.1}
Installing Microsoft.Graph.Groups version {2.26.1}
Installing Microsoft.Graph.Beta.Groups version {2.26.1}
Installing Microsoft.Graph.Planner version {2.26.1}
Installing Microsoft.Graph.Sites version {2.26.1}
Installing Microsoft.Graph.Users version {2.26.1}
Installing Microsoft.Graph.Users.Actions version {2.26.1}
Installing MSCloudLoginAssistant version {1.1.38}
Installing ReverseDSC version {2.0.0.27}

(and it also uninstalled older versions of the above).

I then ran the command 'Update-M365DSCModule' in PS7 and here I got the following messages:

WARNING: The dependency {ExchangeOnlineManagement} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Applications} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Applications} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Authentication} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.DeviceManagement} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Devices.CorporateManagement} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.DeviceManagement.Administration} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.DeviceManagement.Enrollment} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.NetworkAccess} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Identity.DirectoryManagement} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Identity.Governance} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Identity.SignIns} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Reports} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Search} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Teams} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.DeviceManagement.Administration} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.DirectoryObjects} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Groups} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Beta.Groups} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Planner} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Sites} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Users} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {Microsoft.Graph.Users.Actions} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {MSCloudLoginAssistant} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.
WARNING: The dependency {ReverseDSC} does not support PowerShell Core. Please run Update-M365DSCDependencies in Windows PowerShell.

I have since run Update-M365DSCDependencies in Windows PowerShell but still getting the above messages in PS7 when I run 'Update-M365DSCModule'

[FabienTschanz]

The manifest you are referring to is a little bit older than what was just released yesterday. You'll always find the latest released manifest on the master branch of this repository.

Did you restart the PS7 session after installing in PS5.1? I don't receive the messages you mentioned.

Don't worry about the warnings, they're just there so that you won't be installing those modules in the PS7 environment. Otherwise, it won't work again if you use e.g. Start-DscConfiguration in PS5.1.

Installing the latest version and uninstalling older modules is part of the Update-M365DSCModule cmdlet. No way around it. It installs and enforces only the modules listed in the Manifest.psd1.