Tuesday, August 20, 2013

SCCM: How to structure Software Updates

I've had some real struggles with coming up with a good system for managing software updates in SCCM since we went live back in mid-2012. Its taken over a year with much hair pulling and gnashing of teeth but I think we finally have a pretty decent system in place. The worst part is all the books you buy are very confusing and (quite frankly) they tend to make recommendations that are flat wrong. Maybe they are focusing on theory rather than practicality but I'm just amazed at how badly so many books written by so-called "experts" managed to steer me off course with regards to this key feature in SCCM. I had a real eye opener at a conference recently that made me sort of "un-learn" a lot of notions that had been drilled into my brain based on three separate SCCM 2012 books. I thought I'd share some of that here so (hopefully) others won't have to struggle with repeated "trial and error" attempts to roll out a solid SCCM 2012-based software updates strategy:

Don't split updates up by product name! It’s a waste of time. The client agent on the local machine will determines product categories automatically and only downloads the updates that are needed! Instead separate and name your software update groups by periods of time. Start with 2003-2010 as updates only go back to year 2003 and all the updates up to 2010 should fit in a single package. Subsequent groups should be broken up by individual year 2011, 2012, etc. Keep update groups under 1000 updates or you will have problems. Make updates REQUIRED for Workstations. Make updates AVAILABLE for Servers. Tell application owners it’s their responsibility to install and reboot their servers. Clarify that “No Reboot = Not patched” in most cases! Reboots must be planned when installing updates! At the same time try to identify those servers that can be patched and rebooted automatically without having an impact on your user base.

Every time a software update group changes the agent has to rescan all the content of that group. For this reason you want to keep automatic update groups to a minimum! This is both in terms of the number of automatic update groups you create and the amount of updates they are allowed to contain. Create static (or non-automatic) software update groups to store your updates from previous years while limiting automatic updates to a timeframe of the last month, week or day. Not only does this make updates easier to manage it also reduces the load on both the SCCM server and the SCCM clients as update agents won’t have to constantly parse a bunch of old updates every time new updates are released! To reiterate: Use static groups for older updates and organize by year! Use automatic update groups for monthly and daily updates only!

When you create the packages for your update groups make sure to name them by year as well. You want to use a year-based package name for your automatic monthly updates as well as they will eventually be moved to a year-based group anyway. So automatic monthly updates that take place in year 2013 should target your "All Updates for 2013" package. You should plan to do yearly maintenance where you move all the automatic updates for the previous year into a static group (almost like archiving) while creating a new automatic group and year-based package for the incoming year. Also using one yearly package for all updates means you only have one package to monitor during deployments. This is done in the SCCM console under “Monitoring -> Overview -> Deployments”. The compliance column there is your exact compliance figure for the collection you are targeting (as opposed to the compliance column in the software update groups view which is for ALL SYSTEMS).

Additional tips and items to consider:

  • Microsoft best practice is to patch per month, forget quarterly it’s too complicated.
  • Enable delta updates for Distribution Points to get updates out faster and reduce server load.
  • Create alerts for updates, start with 75% compliance at first. Then move up to 80, 90, etc.
  • Updates need to go out to Pre-Production be 7-14 days out from Patch Tuesday, then wait another 7-14 days before deploying to Production.
  • Create Deployment Templates! These will allow you perform manual software update-related actions very quickly. For example: Have a template to push out day 0 critical updates immediately.


Good luck!

81 comments:

  1. Why would quarterly updates be too complicated?
    Thanks, Al

    ReplyDelete
  2. Hi Al,

    I personally struggled with full automation for quarterly updates. I was recently at a conference and a Microsoft MVP pointed out that Microsoft really intends us to adopt monthly patching so I sort of bailed on a quarterly strategy after that. Do you have a different opinion? I'd love to hear your thoughts.

    ReplyDelete
  3. I was thinking of doing multiple Software Update Groups and one deployment package. Do you think there would be an issue? I know you mentioned less then 1000 updates per Update Group. Does that go for the deployment package as well?

    ReplyDelete
  4. Nope, that will work just fine. In fact I'm doing that now for my monthly delta updates. I have a package named Year-2013 and I have 3 software update groups (October-2013, November-2013, December-2013) that all point to that package along with a master group (Year-2013) for everything else that came out prior to those months (for year 2013) that has already been deployed. When the update groups hit 90% compliance I then move them into the master group (Year-2013) sort of like archiving, but also to deal with any out-of-date computers that come online later that I may have missed in the first push. I choose to break it out by the last 3 months so that I can keep a close eye on the newer updates and it also allows me to stagger them out to deploy at different times in the quarter. That system works for me, but you may have different needs. The point is you *can* point multiple update groups at a single package without issues.

    ReplyDelete
  5. The thing is I want to do all my update groups with a single package. ie. 2007 and before, 2008, 2009, 2010, etc...

    Also what search criteria's are you doing for your update searches, I keep getting a bunch of language packs which are most likely not relevant.

    ReplyDelete
  6. Nah, do it by year. Its not as bad as you think. Doing it by year resulted in only 4 packages for me. Thats because there were just not a lot of updates released between 2003-2010. I was actually able to consolidate all of those into a single package (named Year-2003-2010). After that things seem to explode and I decided it was best to do a single package per year. Its my understanding that you want to break out your packages somewhat for load balancing.

    As for my search settings, I have a single ADR that runs once a month on patch Tuesday. It auto-creates a dated software update group, auto-downloads and auto-deploys "Critical Updates", "Security Updates", "Update Rollups" and "Updates" for several versions of Windows and Office to a series of test VMs. If everything installs okay in testing I manually schedule a production deployment from the group it created. Also I exclude anything that has been superscended along with with any updates that have "itanium" in their name.

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Sorry I accidentally deleted Saulius M's comment. But it read as follows:

      "Hi, how do you deal with expired and superseded updates in your existing deployed software update groups. Do you remove them every month?"

      This question and several others presented here made me think that a more detailed post explaining my overall design philosophy regarding software updates with SCCM was needed. I've posted it here:

      http://www.acupofit.com/2014/01/sccm-updates-go-for-autopilot-not-total.html

      I hope that does a better job explaining how I handle SCCM-based software updates and why I do it that way. Any questions or discussion around this topic is welcome in the comments section here or at under that post.

      Delete
  8. After reading multiple blogs, technet articles and other sources to try and devise my firms software update methodology, I ran across this blog post. You helped me understand everything and put it all together perfectly. Thank you!

    ReplyDelete
  9. Glad to help Dan, I wish I had time to blog more on this stuff. I'm still learning myself and its just a slow-going process. We have several SCCm-related goals on target for this year. Hopefully if I get those done I can post some more on how they went down and what we did. Stay tuned!

    ReplyDelete
  10. Aw, man! This post is a life saver. I'm pretty much learning SCCM from the seat of my pants and while I've maintained WSUS with ease for a long time now, SCCM is a whole other beast when it comes to software updates and is WAY more complicated! The yearly groups are an excellent idea that I didn't even THINK of doing! Thank you! I'm gonna read your other post and see if I can glean some more ideas form you!

    ReplyDelete
    Replies
    1. Yeah a lot of the guides out there that break it up by product really mess up the approach in my opinion. Year-based just makes more sense. The best part is if you add update categories later you can just drop them into their appropriate year-based update groups and *bam* they just go out and install. No fuss no muss. There may yet be a better approach but this one works best for me.

      Delete
  11. I am a beginner with software updates in sccm. It would be nice to have step by step tutorial with pictures.... I probably must read other articles to understand your strategy.. I suppose it would be very beneficial to me once I know more about software updates in sccm

    ReplyDelete
    Replies
    1. The video I posted here helped me a lot:

      http://www.acupofit.com/2013/08/video-sccm-2012-sp1-software-updates.html

      Sorry, I just haven't had the time lately to pout together a post with a bunch of screenshots. maybe I'll do that at some point in the future.

      Delete
  12. Hmm, trying to build a fairly automated patch setup. And come up with this.
    ADR on patch tuesday – Downloads the security and critical patches for our test group.. and stores them in the package.
    ADR 15 minutes later, – uses the same package for storage, but makes the updates availble for preprod 2 days later.
    ADR 15 minutes later again – still same package but makes the updates available 8 days after download, and deploys them to select Prod servers ?
    This way with the buildin delays and maint windows, we should have a fairly hands off setup of updates.. Which currently is only Critical and Security, but could include others as needed..
    On paper it looks good, chances are not extra updates have been relased in the 15 minutes between ADR runs. And the delay in deployments should give us ample to time to react to any probelms.. ?
    Or am I missing something here ?
    - See more at: http://anoopcnair.com/2013/01/18/configmgr-sccm-patch-management-pros-and-cons/#sthash.adgwkE0D.dpuf

    ReplyDelete
  13. Helpfull post, but I am still struggling with the exact setup you have chosen for the ADR and de Software Update Groups. For instance... do you download all updates or just the updates that are REQUIRED? How can you easily tell your SUG selection doesn't exceed 1000 updates?

    Probably still to much thinking in WSUS terms instead of SCCM terms when it comes to updating, but I howto guide would be really helpfull or just some extra pointers :) The video didn't help me in that way.

    ReplyDelete
    Replies
    1. I do all non-expired, non-superseded updates (required or not) for all the product categories we support and break them up into year-based software update groups. Doing it that way has kept us below the 1000 update limit. If you do the same and have more than 1000 updates per group then you may be updating more product categories than we are and you may need to come up with a different system.

      Delete
  14. Hi ZeusABJ,

    I watch the video and read your article. I am a little bit confused, please tell me which is correct.(1 OR 2)

    1. Lets say I have 2003to2010, 2011, 2012,2013 static SUGs. Then I create montly new SUGs for 2014Jan, 2014Feb, 2014Mar....and so on, by using an ADR which creates a new SUG everytime. and when year 2015 comes I make a yearly maintenance and combine all those updates from montly SUGs into All2014Updates SUG. Then I delete the 2014 montly SUGs because They are empty now and their member updates are in All2014Update SUG.

    OR

    2. I create 2003to2010, 2011, 2012,2013 static SUGs and I create a SUG named All2014Updates and create an ADR which adds new montly updates in that existing All2014Update SUG. So I will never have SUGs for months such as 2014jan, 2014Feb...so on.

    ReplyDelete
    Replies
    1. Nevermind, I watched the video again and I understood that option 1 is correct.

      Delete
  15. Cool, glad you figured it out!

    ReplyDelete
  16. Hi, I followed your advice in creating year and monthly based groups but I am not not sure who you actually push these updates out to clients.

    Can you offer some advice on what needs to be be done next?

    Scott

    ReplyDelete
  17. I'm a little late to this thread, but if I have a Deployment Package for software updates for each month of 2013 but wanted to consolidate that to one deployment package for the entire year, how would I go about making this change? I have already moved all 2013 software updates to a single software update group.

    ReplyDelete
  18. Hey Mark. never too late to party right? There are several ways to do this but here is how I did it:

    1) Create a new shared folder on the share where you download your update files.
    2) Create a new package and use your new share as its source.
    3) Create a new year-based software update group (do not exceed 1000 updates).
    4) Edit membership on month-based groups to add them to the year-based group.
    5) Re-download the updates the year-based group (choose your new package).
    6) Clean up all your old month-based update groups, packages and source folders.
    7) Re-deploy your new year-based group to the desired collections.

    ReplyDelete
  19. Hi,

    I have found this post really useful in structuring the updates where I work so thank you!!

    I do have a question though....

    When creating the 2003 - 2010 package for example do you select all the updates available or just the "Required" updates?

    And what about the monthly ADR's after each patch Tuesday? Do they include all available updates for that month or just the "Required" updates?

    I hope that question makes sense?

    Many thanks in advance,
    Nick

    ReplyDelete
  20. Hi Nick,

    It makes perfect sense and the answer is its all about preference. "Required" updates are determined by what is in your environment. Also you have to consider how your environment may change down the road. Just because an update is not currently "required" doesn't mean it won't become required later. Your C++ and .Net Frameworks are a good example of this. Say you never deployed .Net 4.5 in your environment but suddenly need to do so. Well suddenly patches for .Net 4.5 become "required" by a bunch of computers. If you already have them downloaded and staged in a software update group, SCCM just patches them up automatically the moment it discovers .Net 4.5 on a managed machine. So (as you may have guessed from that explanation) we let our ADR download all available updates, required or not every month. Then I go through clean them up a little and schedule them out. I just find it easier that way and it tends to "future proof" my patching.

    ReplyDelete
  21. Quick question. I follow the same process that you mention. However, I do split up SUG's between Servers and Workstations. So we would have a December 2014 Workstation and a December 2014 Server SUG. Then the day before Patch Tuesday we move the previous months updates into the Yearly for both a Server and Workstation. Also, we have SUG's for Exchange and SQL. Would you combine the Exchange and SQL into the Servers SUG? Would you just make a single monthly SUG for both servers and workstations?

    ReplyDelete
  22. Great article - reading all I can as I prepare for Server Patching with maintenance windows. One question, your reference "Every time a software update group changes the agent has to rescan all the content of that group." does that mean all Software Updates Groups are rescanned, or only that are deployed via Deployment Package?

    ReplyDelete
  23. If you have a software update group deployed and you change it your clients will have to reparse (or rescan) the updates. This can result in a CPU spike.

    ReplyDelete
  24. Hi ZeusABJ,

    the only concern i have by doing this is following:
    when someone click on the yearly update packakge and click refresh, do it re-send the entire package down to DPs, it that is true and some engineers does it for whatever reason, our WAN link to DPs would be stuffed.

    Cheer
    Jerry

    ReplyDelete
    Replies
    1. dont worry about it, i figured it out, answer is no. thanks anyway, great article.

      Delete
  25. Thanks for the post!

    How can we clean up the cruft in Software updates from a couple years of poorly managed/organized updates?

    ReplyDelete
  26. Sorry thats a pretty generalized question and a bit out of scope for this post. Not something I can answer without knowing a lot more about your environment.

    ReplyDelete
  27. I am curious as to whether or not you should schedule patching for the previous months updates as to stay a month behind Microsofts or if you should stay current and run the current months updates each month. anyone?

    ReplyDelete
    Replies
    1. I am needing this information for the company i am currently working with.

      Delete
  28. This comment has been removed by the author.

    ReplyDelete
  29. Personal preference, company policies and criticality can dictate this. If there are known zero day exploits in the wild and a patch gets released to plug it, I may push that within an hour of being released. That being said the norm for our company is to patch clients 2 weeks after release and we always roll one month in the past on servers (for stability reasons). I suggest you discuss it with whomever is affected in your company and come up with an arrangement that keeps the systems as up-to-date as possible without disrupting their workflow.

    ReplyDelete
  30. Great information ZeusABJ. Thanks for the reply on this.

    ReplyDelete
  31. Great information ZeusABJ. Thanks for the reply on this.

    ReplyDelete
  32. Great information ZeusABJ. Thanks for the reply on this.

    ReplyDelete
  33. This is great. Thank you ZeusABJ. I owe you a beer.

    Quote"
    "Every time a software update group changes the agent has to rescan all the content of that group"
    I also want to quarterly want to change membership.
    If I edit the membership of monthly SUG to Yearly SUG, will the agents rescan the yearly SUG? Because the yearly SUG is also deployed to the same collection as the monthly SUG. Like you said, "to deal with any out-of-date computers that come online later "
    Hope my question make sense

    ReplyDelete
  34. Yes, if you change a software update group in any way it the agent should re-parse it to pick up the changes. I asked this very question at TechEd/Ignite one year and that was the response I received.

    ReplyDelete
  35. Thanks for the writeup! My history with Software update has been with CM2007 and recently back to WSUS3 in my current role. I haven't touched Software Update in CM in 3 years. I am implementing CM2012 now and need to forget about the whole update process. One question, how in the world do I enable delta updates in the deployment package? Do I need to enable prestage content to take advantage of that?

    Thanks again for the writeup - its helped me tremendously so far!

    ReplyDelete
  36. "Delta Updates" is just a term I use for any updates that are outside your standard year-based update groups. Remember this write up is geared around design theory for software update group structure. Not actual settings in SCCM. Glad you found it helpful.

    ReplyDelete
  37. ZeusABJ - Happy New Year. How do you handle machines that have been off for a while and missed deadlines? Do you use maintenance windows or instruct your users to configure their business hours to not be impacted? I need to start migrating users from WSUS where some have missed updates to CM12. I've setup my SUGs by year and deploying them to my workstation collection. Those deadlines have passed for new machines. Is there anyway to stop them from just rebooting during the day?

    ReplyDelete
  38. Our servers are locked into a maintenance window. They only get patched in the 3rd Sunday of every month between 7-11PM. Clients? We just leave any previously deployed patches active at all times and have them set to install quietly (without user interaction). If a machine is left offline for an extended period of time SCCM just catches it up silently whenever it comes back online, hits its polling interval and runs the software update actions. When patching is done the user is notified via pop-up in the tray that updates were installed and they have 24 hours to reboot their machine.

    ReplyDelete
  39. Zeus,
    I found your blog yesterday and it has helped me immensely. I just stood up my first green field instance of SCCM and have been struggling to wrap my head around a manageable approach to patch management.

    ReplyDelete
  40. Hey Rob. Yes, it can be very confusing. The crazy part is how unhelpful a lot of the books out there are. I got so frustrated I started writing up my own how-to's and decided to stick them online not just for all to see but for me to reference later as needed (lol). I've been pretty negligent around here with new posts and should probably update a few areas. I've recently stood up a new SCCM server running the 1602 update (way to make things even more confusing there Microsoft) and have a lot of new ideas for posts. Just super busy right now and strapped for time. Maybe soon! Either way, glad you found this helpful!

    ReplyDelete
  41. Hi mate,

    Thanks for all info. If we don't create a collection as per product how can we deploy updates to specific collection. For example If I am applying updates yearly updates to Windows 2012 R2 servers and my yearly updates contain all products (Office, Windows 7, ....) So you mean it will only detect required update?

    ReplyDelete
  42. Hi mate,

    Thanks for all info. If we don't create a collection as per product how can we deploy updates to specific collection. For example If I am applying updates yearly updates to Windows 2012 R2 servers and my yearly updates contain all products (Office, Windows 7, ....) So you mean it will only detect required update?

    ReplyDelete
  43. Hi Syed,

    Thats correct, Windows will be able to sort out the updates it needs from your software update groups automatically so its okay to mix and match products in the same group. Again the main goal for my organization with the time-based approach is to keep the update groups under 1000 updates (and it just makes sense as Microsoft releases updates in a schedule). That being said I do have a coupe of 1-off software update groups that are product specific (for example I have some servers that will break if we install .NET Framework 4.6.1 so I have it separated off into its own private SUG that is deployed to a different collection) so there are exceptions. By and large though, putting all our supported product categories in a series of groups based on intervals of time (not product names) works best for us. Interestingly enough I was at Microsoft Ignite last year and attended a session in which they recommended doing software updates that way so looks like I was really on the right track when I wrote this post up a few years back (lol).

    ReplyDelete
  44. So then since you have SUG's broken down into years, do you just deploy every year's SUG to a production collection? for example, do you deploy the SUGs for 2003-2010, 2011, 2012, 2013, 2014, 2015, 2016, and all the ADRs for each month to the same production collection?

    ReplyDelete
  45. Hi Andrew,

    Normally I like to have individual collections for each deployment (it creates a bit of "double-working" overhead but I've found it helps me keep better track of things) however, in the case of the year-based updates I am using a different method. I have three collections

    1) Year-Automatic
    2) Year-Auto-No-Reboot
    3) Year-Manual

    I have all my year based groups deployed to those three collections. I think the names make sense but (just in case they don't) the first is for Automatic install with Automatic Reboot, the second is Automatic install with Manual Reboot, the third is Manual install with Manual Reboot. I also have a 4 hour maintenance window from 7-11 the third Sunday of each month applied to all our servers to prevent something from accidentally going out early.

    ReplyDelete
    Replies
    1. are you only patching client OS? just wondering about Year-Auto-No-Reboot on a server that cant be good?

      Delete
    2. We control our server updates and reboots with a maintenance window. All our servers auto-update and auto-reboot the third Sunday of every month between 7PM-11PM (funny thing thats tomorrow night, in fact).

      Delete
  46. I see. I'm just trying to think this through in my head. If you have your SUG's broken down into groups, wouldn't that 2015 (for example) contain all the updates from previous years all rolled up into it because of superseded/expired updates, etc..? Would you even need to deploy the older year SUG's to those collections?

    Sorry if I'm not making any sense, I'm new to SCCM and trying to grasp how this works. And yes, I'm the Andrew from your other article with the 5000+ client hospital. :)

    ReplyDelete
  47. This is difficult to convey without screenshots (I really need to move to a better blog site) but yes, I keep all the SUGs for previous years deployed. That way if an older machine that has not been patched in a while comes online it will get updated. Also if some "power user" removes an update I don't want removed then SCCM will just reinstall it automatically the next time a software update scan cycle runs.

    As for the expired and superseded updates? I clean those out month to month (even in the older groups). I don't know if you've noticed this in the news yet but Microsoft announced this week that they are shifting to a cumulative update model for Windows 7/8/10 going forward. Under this new model they'd release monthly "Cumulative Updates" that contain all the prior updates as well. How this change in strategy will affect how I do things with SCCM is yet to be determined.

    Again though, the only reason I break things up by years is because Microsoft releases updates with a date stamp every patch Tuesday and (in my case) year-based SUGs have managed to keep me below the "1000 updates per SUG" rule of thumb you should follow for performance. Here's another illustration about how I deploy them:

    SUG Name: Deployment Name: Collection Name:
    Year-2014 -> 2015-Full-Auto -> Year-Based-Full-Auto
    Year-2014 -> 2015-Auto-No-Reboot -> Year-Based-Auto-No-Reboot
    Year-2014 -> 2015-Manual-Only -> Year-Based-Manual-Only
    Year-2015 -> 2015-Full-Auto -> Year-Based-Full-Auto
    Year-2015 -> 2015-Auto-No-Reboot -> Year-Based-Auto-No-Reboot
    Year-2015 -> 2015-Manual-Only -> Year-Based-Manual-Only

    (With year-based SUGS going back as far as 2010). Hope that clears things up.

    ReplyDelete
    Replies
    1. Hi Zeus,

      I'm in the middle of testing out updates based on your methods and I'm having a strange issue. I've deployed all years SUG's to a test collection. All except for the 2014 deployment is working. I've been scratching my head on this for days now and I can't figure it out. All of the SUG's and deployments are identical (except for the content of course), yet none of my Windows 7 test machines will download/install the 2014 updates.

      The only error message I can see in any of the logs is in the WindowsUpdate.log:

      2016-10-05 08:46:19:171 932 b98 Agent WARNING: Failed to evaluate Installed rule, updateId = {189A8F50-0C3A-4FDF-8BC2-BC23A3EB11FB}.101, hr = 80242013

      This update ID points to KB982861 which isn't even in ANY of my SUG's so I'm not sure what's going on. These are brand new Windows 7 VM's.

      The UpdatesDeployment.log sees that there are 313 updates, but it doesn't even bother downloading/installing them. There are no maintenance windows or anything and the deployment is configured for "Required - As soon as possible."

      Have you ever seen anything like this?

      Delete
    2. Hi Andrew. Unfortunately no, I have not encountered this issue. When I'm stumped I usually find my answers here:

      https://social.technet.microsoft.com/Forums/en-us/home?category=systemcenter2012configurationmanager

      Delete
  48. Currently we have 2 ADR's. They run a week a part so that we can "test" updates on certain machines. Each one of those creates a new SUG every month. I'm guessing we'll see a performance increase (or at least lessen the stress on the server) if we move to your model? I did try to create a new SUG but can't figure out how. A google search showed I have to use WMI with powershell to do it? Is that how you do it?

    ReplyDelete
  49. Windows N00b has a great article on this with screenshots:

    https://www.windows-noob.com/forums/topic/4467-using-sccm-2012-in-a-lab-part-6-deploying-software-updates/

    ReplyDelete
  50. This comment has been removed by a blog administrator.

    ReplyDelete
  51. Hi, so I'm curious, now that Microsoft have changed patching to Cumulative updates only, all of this would now technically be moot after Dec 2016 right? We should be able to compress this all down to 1 static SUG for every update up to the end of Dec 2016, then use a monthly ADR to use an existing package as Cumulative updates will supersede previous ones. Thoughts?

    ReplyDelete
    Replies
    1. Yes I've thought about this as well (as implied in my above 'August 19, 2016' reply) to a question posed by Andrew. It would be wonderful if that were the case but (so far) most of the updates in my older SUGs have not ben expired by Microsoft so I have left them in production. You also have to factor in all the other products serviced by Microsoft Update beyond just the OS (in our case Office, SQL Server, Dynamics CRM, etc). This may change at some point but (for now) I'm finding that my organizational structure is still needed (for us at least). The main point is keep your SUGs under 1000 updates. Thats the whole point of all of this.

      I guess as with all things SCCM-based, the best solution probably depends on the environment in question.

      Delete
    2. Agreed, It all depends on the environment. In my situation its all Windows 10, Office 365 and Windows Defender, All of which are Cumulative based updates now, so I would assume it would be safe to deploy in the manner I described above so long as we have a Static SUG with all previous updates prior to the Cumulative update strategy being made official. I do also include the Required >0 condition for updates so it covers Windows 10 machines that are being rolled up to the next release via a Deployment Ring.

      Delete
    3. Let me know how it works out. I bet (even with a simple environment like that) you still wind up needing some sort of system to break up the updates. I'm currently working on getting Windows 10 deployed on over 700 machines myself as well so maybe we'll both find out soon enough.

      ;)

      Delete
    4. I manage a 10k client environment... mostly win10...

      I have 3 basic sugs. Apps,workstation and server. 2 copies of each... one for current month and the other previous.

      This keeps me under the 1k limit easy... usually around 500 each.

      I then deploy apps+workstation to all workstations and apps+server sugs to servers.

      Depending how controlled your environment and imaging is you could cut older updates and redeploy if they are ever needed again.

      Delete
  52. So after a month of testing, here's the results:
    If you are doing a simple deployment such as to all workstations / servers etc. then using 1 deployment package / Static SUG works fine. the behaviour is pretty much exactly like deploying SCEP / Defender updates. the SUG will only ever have a handful of updates while older ones are expired and should be automatically cleaned up by SCCM eventually. ADR will run, SUG / Package will update with new patch list, deployment will commence.

    However, If you are going to do a staged deployment, eg: Pilot Workstations then Prod Workstations or Dev Servers then QA then Prod via configuring additional deployments for the ADR, this approach wont work unless you manage to deploy everything prior to the next scheduled run time of the ADR. This is because if you have additional deployments configured, when the rule is run again, the SUG and package is updated which will then deploy to any previous additional deployments that were triggered which obviously isnt good if you are trying to enforce a Dev->QA->Prod deployment method using additional deployments.

    Hopefully that makes sense! based on the above I have configured my environment as per the below:

    All ADRs create a new SUG and use an existing Package. SUGS will still need to be manually deleted as they empty out (patches expire and get removed). My ADR SUGs use a filter by product and where required is more than 0 (to cover multiple branches / channels of the product) with superseded and expired set to No

    Workstations:
    ADR to run on patch tuesday and deploy to pilot workstations +1 week after patch Tuesday to allow time for patches to be pulled back by MS if there are issues
    Additional Deployment to Production Workstations +3 weeks after Patch Tuesday to allow 2 weeks testing in Pilot (I have an extended pilot as with the new cumulative update model you cannot uninstall individual patches, its all or nothing).

    Office 365 Client updates (Current Channel):
    Exactly the same configuration as Workstation deployments

    Servers:
    ADR to run on patch tuesday and deploy to pilot workstations +1 week after patch Tuesday to allow time for patches to be pulled back by MS if there are issues. Additional Deployments to deploy to QA servers +2 weeks after patch tuesday and Production +3 weeks after patch tuesday. The ADR then kicks off on the next patch Tuesday and goes through the Dev->QA-Prod cycle again without affecting previous deployments.

    ReplyDelete
  53. Nice, thanks for the follow-up Matt.

    ReplyDelete
  54. I have a question. when our company was much smaller and had less servers years ago, all of the servers were bunched into one collection and patched at the same time each month, then a separate reboot program published out hours later. What we've noticed is that as the server count has increased over the years (hundreds more) this approach no longer is sufficient. engineers wait up all night long for sccm update reports to refresh and show accurate data as to what has patched and rebooted successfully as opposed to what still needs a reboot. will the reports refresh faster if we patched into smaller collections for servers? whats a good number of servers to have in each collection to patch to? I hope that makes sense.

    ReplyDelete
  55. I'm really not sure I'm qualified to answer that question as the highest server count I've ever had to deal with is 300. That being said I have dealt with client collections in excess of 2000. As for the reporting? that catches up when the database catches up. I usually wait 24-48 hours for my reports to catch up then I evaluate the results and take action is needed. Sorry, I know thats not much help but its all I can give on this subject.

    ReplyDelete
  56. This comment has been removed by a blog administrator.

    ReplyDelete
  57. I am new to SCCM and do not know how to create an archiving deployment job that will run forever. I created a deployment job but it finished and is no longer listed on the SUP.

    ReplyDelete
  58. Hi BarryS,

    Well first there is no such thing as an "archiving deployment job", I just refer to a SUG that contains updates I have previously an "Archive SUG". As for a job that deploys "forever", well any deployment you never delete (technically) runs "forever" so there you go.

    So in a nutshell just create a SUG containing all the updates you have already deployed to all yoru systems and leave it deployed indefinitely. There's your "archiving deployment job that will run forever."

    Easy! ;)

    ReplyDelete
  59. I agree with every line you've written in this post and have been implementing the same methodology on many customer sites. If only I had seen this back in the day, it would have saved me a LOT of head scratching :)

    ReplyDelete
  60. Hi ZeusABJ,

    Fantastic job on this blog post. I was wondering how you manage Server reboots and especially Servers in clusters? Do you manually manage all server reboots or do you have some automation in place?

    Thank you very much in advance
    Craig

    ReplyDelete
  61. Hi Craig,

    Well my company is actually fairly small and we only have a single cluster that consists of two physical servers (everything else is virtual). So I'm probably not the right person to ask about that, but (basically) what I do is set those server to auto-install/manual reboot. Once our deadline hits I wait an hour or so, remote into Server B and reboot it. I then wait for Server B to recover. Once its back up I remote into Server A roll the services to Server B, Reboot Server A and wait for Server A to recover. The next month I flip that process. You probably have a more complex environment than that. If so I'd recommend looking into a recent SCCM feature called "Cluster-Aware Updating (CAU)". I think it only works with Server 2012 R2 and higher so if you are on anything earlier than that it may not be an option, but (point being) SCCM has some options for you on that front.

    ReplyDelete