Troubleshooting packages that just won’t install on DP’s

*** Updated to improve the reliability of the “If all else fails…” section ***

You may have seen in my earlier blog post instructions on how to deploy many packages to DP’s where you know they have a small link or where you’ve had to limit the rate used by SCCM to a very small %. Even where you have followed them, you’ll still probably find some stragglers that never get to the DP without a little bit more….persuasion!

Below you’ll find the process I use to clean up that active packages report:

Step 1 – Identify problem packages

  1. Firstly open up the active packages report in SCCM reporting and sort by DP, so you have a list of all packages that are waiting to install on each of the DP’s. Choose a specific DP to troubleshoot.
  2. Check the sender.log on the parent primary, to see if anything is being sent to the DP. If it is, then files are still being sent to the DP and you should leave this DP for now and move on to the next.
  3. RDP on to the DP you want to troubleshoot (if the server that hosts the DP is just that and not a Pri/Sec site, then you will need to RDP to the server that is the site server rather than just the DP)
  4. Check the root of the drive that SCCM is installed on, for any “_XXXXX.TMP” folders as this will indicate if the DP is still working on uncompressing the PCK files. If you see any folders similar to the above then leave this DP for now and move on to the next. If you don’t, then a good final check for whether the server is doing anything, is to open up the distmgr.log file (I usually open this up in Trace32 as it makes much easier reading) and if you see the message:
    Used 0 out of 3 allowed processing threads.
    then chances are its not going to uncompress any more without a little help from you.

Step 2 – Refresh the packages

  1. Before you do anything else, just try to simply refresh the packages on the affected DP. When doing this ensure the parent primary site is the “owner” of the package for the secondary site DP you are troubleshooting. If you are unsure, connect to the SCCM console of the parent primary site of the affected secondary site and go to the affected package > Distribution Points > if the secondary site DP has a padlock next to it then the “owner” will be another site. To change it go to the SQL server where the SCCM DB is located, open up SQL Server Management Studio (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing) and create a new query for the SCCM DB.
  2. Run the following query, ensuring to fill in your own information:
    update PkgServers set SourceSite=’parent primary site code’ where SiteCode=’affected secondary site code’ and PkgID=’affected packageID’
    So if your parent primary site code was XXX and package XXX00001 has a padlock for secondary site code YYY, then you would run the following:
    update PkgServers set SourceSite=’XXX’ where SiteCode=’YYY’ and PkgID=’XXX00001′
  3. Once done, go to the affected package and right click over the Distribution Points node > select refresh and the padlock should have disappeared.
  4. Now simply refresh the package for secondary site DP and monitor the distmgr.log file on the secondary site server for a short time.
  5. If the package is successfully uncompressing, you will see the entries similar to following:
    Start updating the package XXX00001…
    Attempting to add or update a package on a distribution point.
    Validating the compressed file D:\SMSPKG\XXX00001.PCK for package 0GL000DC
    Used 1 out of 3 allowed processing threads.
    The package file requires a NTFS drive to decompress to.
    Decompressing package D:\SMSPKG\XXX00001.PCK to D:\_S Mq0j8.TMP
    then after a while:
    Created directory D:\SMSPKGD$GL001B107045a9-91fe-4a47-aaf3-3a8c0cfe5b59
    Copying D:\_S Mq0j8.TMP\XXXXX to D:\SMSPKGD$\XXX00001\XXXXX, OK
    Once its starts the copying you can generally be sure the package will uncompress and get removed from the active packages report
  6. If after a reasonable amount of time the package doesn’t start to uncompress or you see that the package ID has been processed but for some reason hasn’t uncompressed move on to “Step 3 – PreloadPkgOnSite tool”

Step 3 – PreloadPkgOnSite tool

  1. If the refresh hasn’t worked you should now try the PreloadPkgOnSite.exe tool to uncompress the outstanding packages. To do this copy the PreloadPkgOnSite.exe tool (can be found online for SCCM) to the server (I usually copy it to %ConfigMgrInstall%\bin\i3860000409), open a command prompt (as admin if using WS2008) and browse to the folder location where PreloadPkgOnSite.exe is saved and use the following command:
    PreloadPkgOnSite.exe XXX00001 (or what ever the package ID is)
  2. A lot of the time, you will see:
    D:\ConfigMgr\bin\i3860000409>PreloadPkgOnSite.exe XXX00001
    ****** Preload Package On Site ******
    Forward package status for pkg XXX00001 to site XXX
    ****** Successfully set the Compressed Package Path on this site ******
    ****** Successfully forwarded the information up the hierarchy ******
    This generally means the package is now going to be uncompressed and the information will be forwarded up to the parent primary and removed from the active packages report.
  3. Again you can confirm if it is successfully uncompressing by following the distmgr.log and looking for the same entries as Step 2 > Action 5
  4. If it doesn’t uncompress or you receive an error when running the PreloadPkgOnSite tool then move on to “Step 4 – Other problems”

Step 4 – Other problems
The above works a lot of the time but sometimes you will find other error messages are returned. See below for fixes I have previously used, for some of these.

Problem 1
Either of the following messages are returned by the PreloadPkgOnSite tool:
Failed to get the specified package XXX00001 in the database. Please check if you have instance rights to that package.
OR
The compressed package path for package XXX00001 is already set in the database

  1. Ensure the PCK file for the package has been copied across to the SMSPKG folder.
  2. If not, copy across manually (from another site) and then run PreloadPkgOnSite tool again for that package. If it returns successful then it will normally begin to uncompress.
  3. If it doesn’t, go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
  4. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so on its direct parent primary site, open up SQL Server Management Studio and create a new, compltely blank View for your site DB.
  5. In the view, copy the below:
    Select * from PkgStatus where SiteCode = ‘Site Code of DP with troublesome packages’
  6. Scroll to the row where the ID matches the package ID and change the SourceVersion to 0 (in some cases there may be more than one row for each ID – change both)
  7. Refresh the package for that specific DP (following the instructions in step 2)
  8. After refreshing the package or packages, new PKG files should get sent to the %ConfigMgrInstall%\inboxes\distmgr.box.
  9. If, after the new PKG files appear, the package doesn’t automatically begin to uncompress then run PreloadPkgOnSite tool with the package ID

Problem 2
When running the PreloadPkgOnSite tool it states as successful but in distmgr.log you recieve the following:
The compressed files for package XXX00001 hasn’t arrived from site 0GL yet, will retry later.
Sent package server list to parent site XXX for package XXX00001
StoredPkgVersion (4) of package XXX00001. StoredPkgVersion in database is X.
SourceVersion (4) of package XXX00001. SourceVersion in database is X.

  1. Ensure the PCK file for the package has been copied across to the SMSPKG folder.
  2. If not, copy across manually (from another site) and then run PreloadPkgOnSite tool again for that package. If it returns successful then it will normally begin to uncompress.
  3. If it doesn’t, go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
  4. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so on its direct parent primary site, open up SQL Server Management Studio and create a new, compltely blank View for your site DB.
  5. In the view, copy the below:
    Select * from PkgStatus where SiteCode = ‘Site Code of DP with troublesome packages’
  6. Scroll to the row where the ID matches the package ID and change the SourceVersion to 0 (in some cases there may be more than one row for each ID – change both)
  7. Refresh the package for that specific DP (following the instructions in step 2)
  8. After refreshing the package or packages, new PKG files should get sent to the %ConfigMgrInstall%\inboxes\distmgr.box.
  9. If, after the new PKG files appear, the package doesn’t automatically begin to uncompress then run PreloadPkgOnSite tool with the package ID

Problem 3
You receive the following message in the distmgr.log file:
The compressed package file D:\SMSPKG\XXX00001.PCK is either missing or corrupted, CTool return code = 11

  1. Delete the current PCK file for the package that is having this problem from the SMSPKG folder. Copy a new version of the PCK file from another location.
  2. Once copied, run PreloadPkgOnSite tool with the package ID

Problem 4
You receive the following message in the distmgr.log file:
A newer version (1) of package XXX00001 has already arrived, delete the replication file D:\ConfigMgr\inboxes\distmgr.box\INCOMING\RFVI26N1.PKG for version 0

  1. Go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
  2. Refresh the package for that specific DP (following the instructions in step 2)
  3. If the package doesn’t automatically begin to uncompress after a refresh then run PreloadPkgOnSite tool with the package ID

Problem 5
You receive the following message in the distmgr.log file:
The compressed files for package XXX00001 hasn’t arrived from site XXX yet, will retry later.

  1. Ensure the PCK file for the package has been copied across to the SMSPKG folder. If not, copy across manually (from another site).
  2. Run the PreloadPkgOnSite tool with the package ID.

Problem 6
The following message is returned by the PreloadPkgOnSite tool:
Failed to get the compressed file of package XXX00001

  1. The PCK file is missing from the SMSPKG folder. Check if this is the case and if so, copy across manually (from another site).
  2. Run the PreloadPkgOnSite tool with the package ID.

Problem 7
The following message is returned by the PreloadPkgOnSite tool:
Cannot connect to the inbox source, please try again after 5 mins. Failed to get the specified package XXX00001 in the database. Please check if you have instance rights to that package.

  1. Chances are this is a Server 2008 machine and so the Command Prompt must be run as administrator
  2. Right click over “Command Prompt” in Start Menu and select “Run as administrator”

If all else fails…
If you’ve done all of the above and it nothing has worked then doing the following should always work:

  1. Delete the current PCK file, for the package you are troubleshooting, from the SMSPKG folder.
  2. Go to %ConfigMgrInstall%\inboxes\distmgr.box and delete the PKG file that matched the package ID
  3. You now need to edit the SCCM DB (***DISCLAIMER***: Do not touch the DB unless you have a full SCCM backup and are happy with what you are doing). I’m presuming the DP you are trying to add these packages to is a secondary site. If so, on all parent primary sites, open up SQL Server Management Studio and create a new query for your site DB.
  4. In the query, copy the below:
    select * from pkgstatus where SiteCode = ‘Site Code of DP with troublesome package’ and ID = ‘Package ID’
    select * from PkgServers where SiteCode = ‘Site Code of DP with troublesome package’ and PkgID = ‘Package ID’
  5. Double checking to make sure the only results relate to the site and package that you are having problems with, you then need to delete these database entries using the below:
    DELETE from pkgstatus where SiteCode = ‘Site Code of DP with troublesome package’ and ID = ‘Package ID’
    DELETE from PkgServers where SiteCode = ‘Site Code of DP with troublesome package’ and PkgID = ‘Package ID’
  6. Once the delete command has completed, you now need to redeploy the package to the DP. Go into the SCCM console and check the DPs for the package – you should see that the troublesome DP is no longer there. Re-add the DP to the package.
  7. As the entries have been removed from database, SCCM thinks it is deploying the package to a brand new DP and will send the PCK file using the normal method. Watch the sender.log to monitor the package being sent.
  8. On the DP open the distmgr.log and wait for the package to be processed.
  9. If you see the package get processed but not uncompressed, simply refresh the package for that specific DP again.
  10. After refreshing the package, the software should uncompress automatically.
Advertisements

22 Responses to Troubleshooting packages that just won’t install on DP’s

  1. Dan Richings says:

    I find myself constantly coming back to this article when troubleshooting package distribution. I’ve found it to be a truely valuable resource. Thanks for posting and keep up the good work.

  2. Rod Alaminos says:

    Hi Niki, same like Dan. This is a very usefull compilation which is helping me a lot troubleshooting PCKs distribution. Thanks a lot for this.

    Regards from Madrid!
    Rod.

  3. nivor says:

    Hey Rod! Hows things over there?

  4. Rod Alaminos says:

    I’m back here again! hehe.

    All fine, thanks Sir. I hope everything goes well there.

    Cheers
    Rod.

  5. Pingback: Troubleshooting packages that just won’t install on DP’s (via ConfigMgr in the real world) « t3chn1ck

  6. Pingback: Need help preloading packages onto distribution points « MS Tech BLOG

  7. Kiran says:

    What should we do if .PKG file is not arrived to %ConfigMgrInstall%\inboxes\distmgr.box though after refresh DP or delete and Add DP to the DP List.

  8. Pingback: Troubleshooting packages that just won’t install on Distribution Points | CMT Webdesign

  9. Pingback: Troubleshooting packages that just won’t install on DP’s | SCCM

  10. Anoop's says:

    Reblogged this on Anoop's and commented:
    SCCM / Configmgr – Package replication troubleshooting guide

  11. wancho says:

    Hi,
    I have a primary and 15 standard DPs across site (not a secondary, just server share DP), is it possible to preloadpkgonsite even if i just have standard server share DP?

    Thanks!

    • nivor says:

      Hi, so you have 15 DPs under 1 Primary site? It this case you should only have one location where the PCK files go to (one SMSPKG folder) for all 15 DPs. You simply need to run the preloadpkgonsite tool on the Primary Site server and the packages will uncompress on all the other DPs automatically.

  12. seb says:

    very nice post, I had recently a similar problem where a DP didn’t want to copy a package for whatever reason, even manually or using PreloadPkgOnSite, finally I had to change the package source version by modifiying the properties (just adding a space or a . in the comment field will do) and then “update all DP with a new package version”.
    Of course all DP will download the new version so it may not be good for big packages.
    (my config is 1 primary site and 53 secondary)

  13. Arnab says:

    Really very helpfull post for troubleshooting package copy issue. Appriciate the effort taken. Cheers

    Arnab

  14. Sam Robinson says:

    I’d like to add this; If you run PreloadPkgOnSite and you start getting messages to the effect that a newer version of the package has already arrived (reason for this is covered at http://myitforum.com/cs2/blogs/jmarcum/archive/2010/05/19/146968.aspx), look at StoredPkgVersion in the SMSPackages table and check vs the StoredPkgVersion on the source site. If they’re out of whack, just modify the destination site’s StoredPkgVersion and Update that DP again.

    Sam R.

  15. Mannam says:

    I have re installed my DP and copied all the .pck files into DP via USB drive, How do I extract all these packages in the DP? I don’t want to extract one by one with StoredPkgversion.exe, I just wanted to do it with one script or one command.

  16. Pingback: Anoop's » SCCM / Configmgr – Package replication troubleshooting guide

  17. Pingback: Content distribution troubleshooting | sccm road

  18. Amol Bhatt says:

    Thank You ….This resolved my Issue. We were struggling since 3 months and those 10 steps resolved my Problem …Many thanks

  19. Pavel Yurenev says:

    I wouldn’t suggest to do any DB changes without Microsoft Support engineer permission.

    I mean especially this stuff:
    2.Run the following query, ensuring to fill in your own information:
    update PkgServers set SourceSite=’parent primary site code’ where SiteCode=’affected secondary site code’ and PkgID=’affected packageID’
    So if your parent primary site code was XXX and package XXX00001 has a padlock for secondary site code YYY, then you would run the following:
    update PkgServers set SourceSite=’XXX’ where SiteCode=’YYY’ and PkgID=’XXX00001′

    • nivor says:

      Then don’t, that’s why the disclaimer is there 🙂 If you aren’t confident doing it without MS support, then put in a ticket first. Completely up to you however, this does work if you follow the instructions correctly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: