Random errors installing Package in our deployment

[devmcg03]

*Someone at HedgeHog setup our deployment process for us but suggested adding an issue here to help resolve this error. We are on Sitecore 8.2 update 3 and using version 1.09 of the SitecorePackageDeployer.

We currently are running Octopus to deploy to our Sitecore CM and then to our three production servers. Previously we were running into an issue where the SitecorePackageDeployer wasn't even running and someone (from HedgeHog) suggested we append '?force=1' to our Octopus step that runs the Start Package Deployer (/sitecore/admin/StartSitecorePackageDeployer.aspx). This fixed the issue where it wasn't running but now we've been having an issue where the SitecorePackageDeployer fails and our master database is getting out of wack. We have an autopublish setup to run and once this runs and pushes to our prod db it's causing the site to go down. We are unable to see what is happening to our db at this point since Sitecore is throwing errors and we can't browse the tree. Sometimes redeploying fixes it, sometimes it doesn't.

I've attached the install.log and the InstallationHistory.txt that contains the errors we were seeing. Typically if this fails then in the 'Data\SitecorePackageDeployer' folder there will be a .error package and the .json file will contain a message 'fail' message and the 'DeployHistoryPath' will be blank. Let me know if there is any other information you want me to send.

Install.log
InstallationHistory.txt

[cturano]

Hi,
I suspect the issue is that you have multiple installs running at the same time. This is usually caused by the improper use of the force=1 parameter. This should ONLY be used when you are sure there are no other installs running.
The installer maintains its state in a Sitecore property. If the server is rebooted or crashes during an install, this property can be set to "Installing" when no installation is happening. If this happens, no other installs can be started. The force=1 parameter will reset that value and start an install.
If another install is running and you use force=1, you will get errors indicating the .update file is missing, and also create other unpredictable results. I recommend you stop using force=1 in the step that starts the package deployer and a first step in resolving this issue.

[devmcg03]

Thanks for the info. Like I said, we were having an issue where the sitecorepackagedeployer wasn't even running during our deployment process so the suggestion from Hedgehog was to use the force flag. I will remove it though you might hear from me again if it reverts to use again having the issue with it not running. Can you point me to that Sitecore property that you mention where the installer maintains it's state. I'd like to keep an eye on this while I'm testing out these updates.

[cturano]

The property is in the core database. Its called:

SPD_InstallerState_[machine name]

The enumerator for the values is:

    internal enum InstallerState
    {
        Ready = 0,
        InstallingPackage = 1,
        WaitingForPostSteps = 2,
        InstallingPostSteps = 3
    }

You should also see messages in the log that look like:

ManagedPoolThread #9 10:47:10 INFO Setting installer state to InstallingPackage
ManagedPoolThread #9 10:47:10 INFO Setting installer state to Ready

This indicates that the installer is polling the deploy folder for files and is functioning correctly.

[devmcg03]

We are searching for that item in core and it's not returning anything. What is the full path to the item?

[cturano]

It is in the property table in the core database. Here is the code from the Git repo that accesses it:

    internal static InstallerState GetInstallerState()
    {
        Database coreDb = Database.GetDatabase("core");
        return (InstallerState)coreDb.Properties.GetIntValue(INSTALLER_STATE_PROPERTY + Environment.MachineName, (int)InstallerState.Ready);
    }