Snippet from log:
(Jul 30 2018 04:38:04.152 PM -0600),INFO,com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask,
PROGRESS EVENT {Description=[com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask.description],Progress=[70],Detail=[com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask.validateTrueSightPServerSSOConnection]}
(Jul 30 2018 04:38:05.618 PM -0600),INFO,com.bmc.install.product.truesightpserver.TrueSightUpgradeFileCopyImplementation,
LOG EVENT {Description=[Remedy SSO status code 401],Detail=[validateRSSOParams()]}
(Jul 30 2018 04:38:05.634 PM -0600),INFO,com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask,
PROGRESS EVENT {Description=[com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask.description],Progress=[100],Detail=[com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask.complete]}
(Jul 30 2018 04:38:05.634 PM -0600),CONFIG,com.bmc.install.task.InstallationPropertiesHelper,
LOG EVENT {Description=[SET PROPERTY VALIDATION_ERROR_RESULT],Detail=[product.truesightpserver.TrueSightPServerInstallerMessages:RSSO_TEST_CONNECTION_FAILED]}
(Jul 30 2018 04:38:05.634 PM -0600),INFO,com.bmc.install.product.truesightpserver.TrueSightPServerRemedySSOInputValidationTask,
LOG EVENT {Description=[Validation error],Detail=[Failed to connect to the Remedy SSO server.]}

Ensure that no special characters are used in the RSSO Admin user's password.  For example, "!" has been known to cause connection problems.

Defect drtry-6 has been opened to track TSPS 11.0 installation connection problems to RSSO 18.05.
Until that defect is closed, you can use TSPS 11.3.01 to connect to RSSO 18.05.