Stop the "Windows update madness" on WS2016!
Dear Microsoft Windows Server team. We would like _complete control_ over when we would like our our WS2016 to install windows updates! No more slow / timeconsuming server restarts because of "Getting Windows ready. Do not turn off your computer" messages when we just would like a quick restart of a production server.. No more TiWorker.exe using 100% of a CPU core during "*********" and so on.. And please fix the issues with '2017-08 Update for Windows Server 2016 for x64-based Systems (KB4035631)' asap. Thank you.
Hi John, I cannot agree there is any improvement on my lab.
I've tried following:
2016 1607 LTSC GUI patched to 2019-02, incl. SSU - not ok
2016 1607 LTSC Core patched to 2019-02, incl. SSU - not ok
2016 1803 SAC Core patched to 2019-02, incl. SSU - ok
2019 1809 LTSC Core patched to 2019-02, incl. SSU - ok
2019 1809 LTSC GUI patched to 2019-02, incl. SSU - ok
all trying to update to 2019-03. not ok mean - not improvment in speed compared to my previous benchmarks / comparisons.
another thing that I noticed while comparing. During "download", when it is really downloading, there is no network activity in the VM taskmanager in 1607 LTSC GUI only.
2019 LTSC GUI shows correct network activity while downloading.Imho 2016 is "limited" in many aspects. Windows Update is just one of them.
I will recap the situation with the 2019-03 updates. All VMs I've measured had previous updates. I currently doubt there will be any improvement but I really want to double check your findings.
Yesterday updated 2016 1607 2x GUI 2xCore at a customer 4 VMs from 2018-08 to 2019-02 via Windows Admin Center. The process took about about 45 Minutes including restarts
Hi guys, I did some testing too with W2016 GUI boxes too and I experienced there is huge difference between the clean install update and when you work with updates aplied after Jan 2019.
As I have tested it, it took about 6 hours to make a first update with .iso image MLF_X21-3035 downloaded from Microsoft and used for a clean install. But since applied patches in the end of Jan it's just a matter of minutes to update the install and it's very fast (Feb 2019 updates took nearly 7mins to apply and WinDef daily updates are just a matter of 1-2 mins without restart - using sconfig or UI it doesnt matter - i am testing it behind FW + proxy and other HTTP protections on the way).
Now I am working on W2019 Core and UI tests so hopefully will get some numbers too.
" a fix to the current Service Stack might have fixed it in those who are still using 1607 if I'm reading what you said, though on reread it seems this only fixed a DL and not the install/reboot."
Currently there is no fix. It can happen though on fresh installs that the display of download / installing phase are displayed correctly without affecting the total time.
"Do you have more information on the procedure you were using to update the OS of the existing 1607 boxes to 1809? A link to documentation or perhaps what exactly to look for?"
There is no documentation for this as this does not work. GUI <> Core switch is no longer possible (was with 2012R2 but got removed with 2016).
If you need GUI and have a license or active SA for CALs and Server you can easily mount your 2019 ISO and do an inplace upgrade. (Backup / temporarily snapshot in backhand)
Ironically this upgrade wents nearly faster than any monthly CU installation ;)
Not telling there won't be a fix at all in future - while I doubt it - as it disagrees with the LTSC terminology - but all efforts to get MS involved directly failed so far.
Maybe being a GOLD partner is not enough.
Thanks for everyone your feedback and appreciations of my works. It helps a lot get equal the investments of free time into this topic.
"Not seeing a 180* version of 2016 with GUI at all. :( So guessing we either need core instances or start using 2019. :/"
Yes it is like this it is definitely fixed in Server 2016 1809 Core, which means there is no transition from GUI to Core without reinstalling + you may have limitations with apps - lesser of these with 2019 Core (due to Feature on Demand package).
However you need an active Software Assurance for the Server 2016 Core 1809 as it is an SAC branch. you may not use it without active SA. On top SAC Server Core branch has only 18 months support but inplace upgrades are the way to get up just as with GUI from 2016 > 2019 etc.
tl;dr So either you can transist to 2016 or 2019 Core with Software Assurance or use 2019 GUI to workaround this issue.
We have had IT press coverage already and I am trying to get more involved.
Mike Estes commented
Ah, I think I misunderstood what you said and what is available. Not seeing a 180* version of 2016 with GUI at all. :( So guessing we either need core instances or start using 2019. :/
Mike Estes commented
Hi Karl, thank you for taking the time to investigate this issue, was one of my tasks to finally get back to as this has gotten much worse over time. This last patch Tuesday was a nightmare. Probably due to having more boxes on that version of Server 2016.
What's the TLDR? I see you mention in an earlier suggestion to update the OS to a newer version 1809, but it sounds like a fix to the current Service Stack might have fixed it in those who are still using 1607 if I'm reading what you said, though on reread it seems this only fixed a DL and not the install/reboot.
Do you have more information on the procedure you were using to update the OS of the existing 1607 boxes to 1809? A link to documentation or perhaps what exactly to look for?
John / EFD no need to argue. Please check my stats. The issue is any Windows 10 / Server 2016 1607 alone. Any later does not have any issues with updating and Performance.
Belongs to Server 2019 GUI / Core (1809) as well as Server 2016 Core 1803 etc.
2 EFD - having no internet connection at the time of Azure & AWS and hybrid infrastructure? Sorry but its 21st century and advanced firewalling is the solution for today not to be closed in a bunker. And when I compare Linux machines, clean installs, updates with W2016/2019 it's like a heaven and ****. In case of Linux it's transparent, all infos & progress provided to admins every second and all update process takes a few minutes. In case of W2016 I had been working on clean installs this weekend and I saw rolling icon & don't turn of you computer for many hours. However W2016 is fast and stable system but WU it's something horrible. And SCCM has nothing to do with it because WU is internal and not the orchestration around it and SCCM. There can be just improvement like using WSUS and some internal issues W10/W2016 have with searching for online update MS services and downloading http .cab crazy stuff. The new WU is a replacement of a previoud Update services that W10/W2016 provide and this is the crazy stuff that Microsoft introduced with W10.
Also, what are your updates scoped to? Are you installing critical CVE patches or everything? I can assure you that installing anything other than CVE and security patches is the wrong way to go so that may be why we dont see this issue. We dont install anything else, there is no point
I'm curious to know what you are using as an update platform? We have no issues on server 2016/2019 with updates. We are using SCCM, but then again our configuration is fine tuned and our servers have zero internet connection, which should be the standard for any corporation looking to secure its infrastructure
Server 2016 updates = TOTAL NIGHTMARE !!!
Stop this Horror Please!!!
Ad Admin commented
NO! Please DO NOT fix that!
Back when I was student, I made a lot of money by maintaining Windows computers.
Bugs like this and all the other reasons why I don't use Windows made it a very good business.
People who really need their own devices for making money are not using WIndows as their main-operating-system anyway.
Oliver Richter commented
YES stop this update Horror on WS2016 asap!!!
TrustedInstaller/Tiworker with 50-100 % utilization
Restart Durations over 30-40 min. without visible reason.
This is terror for the hard working IT admin.
It cannot be so difficult to solve this problem!!!
WS2016 updating is sooooo long.
And troubleshooting (read logs) is soooo nasty compare to WS2012R2.
Please fix it.
Another example. It is in the code. Proven.
Patching Server 2016 LTSC Core 1607 vs Server 2016 SAC Core 1803 + 2 installations from scratch (each about 3 minutes)
Watch yourself: https://1drv.ms/v/s!ApTx3d3fhinPgpk-2l8Vku1C26Pz7A
Server 2016 LTSC Core 1607
Size 1393,5 MB
Search Updates > 13 sec
“downloading” Updates > aborted after 28 minutes
Server 2016 SAC Core 1803
Size 835,8 MB + SSU 1,2 MB + Delta Update 287,6
Search Updates > 19 sec
Downloading Updates > 2 min
Installing Updates > 2 min 45 sec
Reboot / Applying Updates / logon > 48 sec (typo password measured 57)
Total Patch Times > 6 min 56 sec
Server 2016 LTSC shows the usual behaviour to report download while already installing the update. The status is not consistent with the actual process. This is valid for GUI and core and was fixed later in W10 1703 / Server 2016 SAC 1709
As we see now the process is broken by design, not by update sizes as abbodi86 stated.
The Server 2016 SAC 1803 has quite similar update speed as Server 2019
Despite the package size of 2019 is even smaller.
I don’t expect 2016 1607 Core to finish before minute 25 as last time.
If I am able to install a server in a time of 3 minutes the system cannot be powerless enough to patch anything within a reasonable time.
Just for fun we could install a second 2016 server from scratch to make the sillyness of this bug obvious.
I could literally setup 15 servers + patch them – in a timeframe where one of these 2016 1607 would be completed patching, or more depending if GUI or not.
Showing this a complete clean installation should remove all doubts the issue is by design, and MSFT has fixed it in later code.
you see the new 2016 server 1607 suffers even more issues. longer update search etc. so the 02-2019 and older SSU improved it – except the installation time and wrong display of “download” phase
ok at least there seems to be a relation of SSU and download if no older SSU was installed – interesting enough
KB3192137 – mysterious…
nevertheless even the new installed 2016 has the same installation speed, despite the download issue appears to be fixed.
seems the 2016 LTSC is crashed…
0 percent CPU makes me worried
But I think you get the point, the new installed 1607 isnt that fast either.
I will stop it here. not worth the time.
We’re a small non-profit with about 20 PCs and one physical Windows 2016 server. Updates that used to take an hour or so on a Friday evening with Server 2008R2 now take an entire day or longer with Windows Server 2016. Most recently I tried installing KB4480961 (the January 2019 cumulative update)—it ran for nearly four hours, failed, and then took another hour to roll itself back. Ugh.
Not sure what our long-term plan is yet, but a Synology NAS is starting to look better and better. Windows Server 2019 is available to us through software assurance but I’m reluctant to move to a server OS that was only released a few months ago.
Microsoft, please fix!
Same massive issues and problems here with Windows Server 2016 (inside virtual machines with 2016 as well as on the hypervisor side with Hyper-V based on 2016)
Updating 02-2019 2016 & 2019 Core comparison
Server 2019 LTSC Core
Search Updates > 6 sec
Download Updates > 33 sec
install Updates > 2 min 30 sec
Restart / Apply Updates + Logon > 45 sec
Total Patch Time 02-2019 CU > 3 mins 56 sec
Server 2016 LTSC Core
Search Updates > 28 sec
Download / (install*) Updates > 13 mins 52 sec
install Updates > 6 mins 8 sec
Restart / Apply Updates + Logon > 2 min 57 sec
Total Patch Time 02-2019 CU > 23 mins +/- 26 sec