Please feel free to provide feedback or file bugs here.

Compiling large number of DSC MOF files is slow

We use logic to dynamically compose a ConfigurationData structure and pass it to the configuration. This logic executes quickly. Calling the configuration generates more than 5,000 MOF files for unique nodes. The process takes 1.5 hours on modern server-class hardware with 16GB RAM. Also, the MOF files are all created at the end of the process, rather than one-at-a-time throughout the process. This causes high memory usage. Please optimize the PSDesiredStateConfiguration module to generate large quantities of MOF files more quickly. This issue adds significant delay to the DSC pipeline.

19 votes
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)

    We’ll send you updates on this idea

    Ashley McGlone shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    6 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • Arie H. commented  ·   ·  Flag as inappropriate

        @Zuldan: maybe you're doing this wrong then. You need to find bugs at an early stage in a "Dev \ Integration" tests. Not in Prod.

        I'm not saying there's a 100% fault free in Prod that you didnt find in Dev \ Int. However it will happen less and less if you have a decent Unit tests \ Integration Tests \ OVF implemented.

        It shouldn't be a feature of the product (DSC) just because your tests aren't implemented. else its just covering the real issues :)

        If there's something that is deemed "small change to servers", consider maybe implementing it via DSC wasn't the best way. Maybe GPO \ GPP would have been better. I know the line between DSC and Server GPO \ GPP gets thinner and thinner but one of the criteria's deciding when to use each is recoverability and maintainability.

        If you do find yourself having to recreate 1900 mof files every time for a small change, perhaps the answer isn't in increasing the speed of MOF creation (has its limits in terms of resources) but rather what I wrote in my first comment about Diffing DSC scripts and creating a Diff Mof that updates an existing one.

      • Arie H. commented  ·   ·  Flag as inappropriate

        Agree on the part where the MOF file needs to be created immediately after its finished processing that specific node and not at the end of all the X amount of nodes.

        I dont see however a reason to do all 5000 at one go. Its not like you're going to deploy all 5000 at once, this is a no-no in terms of network and potential management nightmare when things go wrong on large scale. Even, if say you have a setting you want to change at the base of each config that affects all of your nodes, you will still deploy this in stages, thus creating the MOF at once is not really matter how much time it takes, if you plan it correctly to match your stage nodes.
        (this might also goes into somewhat "grey" area of what should be in DSC and what in Server group policy).

        I would like to see something like :

        I have a DSC script that created a MOF.
        I have a change to the script.
        I would like to compare the first version to the second version and the result will be just the changes to the MOF I need (think about a DSC script that is 500 resources long...)
        I then apply the "Update" MOF to the node.
        This means lesser payload in terms of network, lesser resources on the node applying the change.
        and better pipeline all along.

      • Zuldan commented  ·   ·  Flag as inappropriate

        Ashley thank you for creating this uservoice. We have the same issue with MOF generation on a mass scale (takes a very long time). 1,900 MOF files is taking roughly 45 minutes for us to generate. Not great when you want to make quick small changes to your servers. Also not great when you find a bug and you need to roll back a servers configuration. That's an additional 45 minutes downtime for the business.

        Resources per MOF: 83
        Powershell version: 5.0

      Feedback and Knowledge Base