Repository navigation
Upgrade Octopus Server template continues after download failure and passes formatting objects to msiexec #1710
Description
Step template
Upgrade Octopus Server
Step version
10
Octopus version
2025.4.10250
Step template parameter inputs
- InstanceName: OctopusServer
- InstallPath: C:\Program Files\Octopus Deploy\Octopus
- SpecificOctopusServerVersionToInstall: 2026.3.15878
What happened
The MSI download failed with:
The request was aborted: Could not create SSL/TLS secure channel.
The step caught the exception but continued as though the download had succeeded. It logged Download completed, stopped the Octopus Server service, and invoked msiexec.
The msiexec process remained nearly idle for approximately three hours. Its command line contained PowerShell formatting-object type names instead of the downloaded MSI path:
msiexec.exe /i Microsoft.PowerShell.Commands.Internal.Format.FormatStartData Microsoft.PowerShell.Commands.Internal.Format.GroupStartData Microsoft.PowerShell.Commands.Internal.Format.FormatEntryData ...
The Octopus Server service remained stopped until the stalled installer was terminated and the existing server version was restored.
Expected behavior:
- A download failure should terminate the step before the Octopus Server service is stopped.
- Exception diagnostics should not become part of the returned MSI path.
msiexecshould receive exactly one valid MSI path.- The step should wait for the installer and fail when it returns a nonzero exit code.
Reproduction steps
- Configure version 10 of the
Upgrade Octopus Servercommunity step to run on an Octopus Server machine. - Set
SpecificOctopusServerVersionToInstallto a version different from the installed version. - Cause the MSI download to fail, for example through a TLS, proxy, certificate, or network failure.
- Run the step.
- Observe that
DownloadOctopusServercatches the exception but does not throw it. - Observe that the step still logs
Download completed. - Observe that the step proceeds to stop the Octopus Server service.
- Inspect the resulting
msiexec.execommand line.
The output contamination can also be reproduced safely in Windows PowerShell 5.1 without running an installer:
- Execute the same
try/catchstructure fromDownloadOctopusServer. - Replace the WebClient call with a deliberate
WebException. - Assign the function output to a variable.
- Inspect every returned object's type.
The result contains five PowerShell formatting records followed by the intended temporary-file string:
Microsoft.PowerShell.Commands.Internal.Format.FormatStartDataMicrosoft.PowerShell.Commands.Internal.Format.GroupStartDataMicrosoft.PowerShell.Commands.Internal.Format.FormatEntryDataMicrosoft.PowerShell.Commands.Internal.Format.GroupEndDataMicrosoft.PowerShell.Commands.Internal.Format.FormatEndDataSystem.String
This occurs because the catch block uses:
echo $_.Exception | Format-List -Force
Format-List emits formatting records to PowerShell's success-output stream. The caller therefore receives those records together with $tempFile.
More Information
The relevant code in version 10 is:
catch
{
Write-Host "Exception occurred"
echo $_.Exception | Format-List -Force
}
Write-Host "Download completed"
return $tempFile
The caller then continues with:
$tempFile = DownloadOctopusServer $selectedVersionToInstall
StopOctopusServer
InstallOctopusServer $tempFile
The installer is invoked with:
msiexec /i $tempFile /quiet | Out-Null
There are two distinct concerns:
-
The original TLS negotiation failure
The precise cause of the TLS failure has not been established. It may depend on the server's WebClient, SCHANNEL, proxy, certificate, or execution-account configuration.
-
The confirmed template defect
Regardless of why the download fails, the template swallows the exception and emits formatting records into the function's return value. It then stops the Octopus Server service and invokes
msiexecwith corrupted input.
The PowerShell 5.1 reproduction produced the same formatting-object type names found in the real msiexec.exe command line, confirming the connection between the catch block and the malformed installer invocation.
After the incident:
- The original Octopus version,
2025.4.10250, was restored successfully. - A separate HTTP request to the Octopus download endpoint returned HTTP 200.
- BITS subsequently downloaded
Octopus.2026.3.15954-x64.msi. - The downloaded MSI was 874,090,496 bytes and had a valid Authenticode signature.
This later successful download suggests that the download endpoint was available. It does not establish the exact cause of the earlier WebClient TLS failure.
Suggested correction:
- Make download exceptions terminating.
- Do not write exception-formatting objects to the success-output stream.
- Validate that the downloaded file exists and is a valid MSI before stopping the service.
- Invoke
msiexecwith an unambiguous path. - Wait for installer completion and validate its exit code.
- Ensure temporary-file cleanup occurs in a
finallyblock. - Add a regression test covering download failure.
Current template source:
https://github.com/OctopusDeploy/Library/blob/master/step-templates/upgrade-octopus-server.json
Activity
bcullman commentedon Oct 17, 2026
Follow-up investigation
Further testing confirmed that the original failure-handling defect is real, but also showed that failing safely is not sufficient for this step to remain usable.
The original
WebClient.DownloadFile()request repeatedly failed after transferring a substantial portion of the installer. The underlying exception reported that the connection was forcibly closed after approximately six minutes.We ruled out several possible causes:
We then tested a compact version of the step that:
Content-Range.msiexec, checks its exit code, and attempts service recovery after an installation failure.During an actual test-server upgrade, the ranged download reached 503,316,480 bytes before one request failed after approximately five and a half minutes. The step retried from the last downloaded byte, completed all 874,090,496 bytes, validated the installer, installed the requested Octopus Server version, restarted the service, removed the temporary installer, and completed successfully.
The proposed pull request will therefore address both parts of the problem:
msiexecarguments.The implementation includes isolated regression tests for range sequencing, retry and resume behavior, invalid responses, length and signature validation, partial-file cleanup, MSI exit codes, service recovery, and the existing template contract.