To improve Windows Server I suggest you ...

2016, WDS, Deduplication: Option to disable RES.WIM usage

A bit of history here. Windows Deployment Services has been around for quite a number of years, and it uses a file-based deduplication mechanism to conserve disk space.

When we capture an image with WDS, what it is actually doing is creating hashes for all files on the disk and then checking to see if those hashes are present in RES.WIM. Any hashed files not present in RES.WIM are copied up to the server, added to RES.WIM, and then the disk image itself is just a list of hashes to pull out of RES.WIM to reconstruct the image.

Without deduplication, it is to our benefit to only create a single huge imaging group, because for each additional image group there is a new RES.WIM which is independent of the other and may duplicate hashed files from the other RES.WIM. Throw everything into it, XP, Vista, 7, 8, 10, and all of them automatically combine into a single huge RES.WIM hash pool that greatly reduces file storage.

Except RES.WIM has a problem in that we can never shrink it, because WDS cannot know what image files we've deleted and what hashed files it needs to keep. Even if we've deleted all our Win XP images, those files for XP will live on forever in the RES.WIM hash pool.

Apparently the only real way to shrink RES.WIM is to create a new image group, export images from the first one, and import to the new one, which takes up even more disk space, before we can finally remove the old image group to recover disk space. It's extremely unhandy and inconvenient.
,

Though, Microsoft has now come up with sector-based deduplication of drive volumes. Scan in the background, deduplicate identical sectors, and regain free disk space.

This accomplishes the same thing as the file-based deduplication of WDS, but this is now even better, because we can delete deduplicated files and we can recover the disk space used by the non-duplicate parts of the image file.

Sure, we can run both WDS and sector-based deduplication on the same drive, but then we still cannot ever shrink RES.WIM, so the benefits of dedupe + RES.WIM are less than just using deduplication alone.

It seems better to just disable WDS RES.WIM file hashing completely, save all images as raw huge file collections in separate WIMs, and then let the nightly optimization shrink the WIM image disk usage to nothing, merging its content with other WIM images previously saved to the deduplicated drive.

If we then delete unneeded images, we will recover disk space as expected, and disk usage will not just grow forever as with the current RES.WIM file dedupe.

Posted on Windows forums here:
https://social.technet.microsoft.com/Forums/windowsserver/en-US/c33ce755-8c02-4f1c-94b9-2ac5cb79c794/2016-wds-deduplication-disable-reswim-usage?forum=winserversetup#c33ce755-8c02-4f1c-94b9-2ac5cb79c794

2 votes
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)

    We’ll send you updates on this idea

    Dale MahalkoDale Mahalko shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    0 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...

      Feedback and Knowledge Base