It's running 6.0
Alright it's back up! Success! After taking a closer look in the VCSA's /var/log/vmware/vpxd/vpxd.log line by line after the SSL certs are exchanged between VCSA and the PSC, the VCSA was complaining that it could not find the SSO Admin server (/sso-adminserver/sdk/vcenter.local) when connecting to the PSC with 404 not found error as seen below:
2018-08-08T17:29:41.070Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=dbPortgroup] [VpxdInvtDVPortGroup::PreLoadDvpgConfig] loaded [9] dvpg config objects
2018-08-08T17:29:41.074Z warning vpxd[7F5E84185700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f5e80e73f10, h:25, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:18090'>>, e: system:111(Connection refused)
2018-08-08T17:29:41.075Z error vpxd[7F5E84185700] [Originator@6876 sub=HttpConnectionPool-000001] [ConnectComplete] Connect failed to <cs p:00007f5e80f162e0, TCP:localhost:18090>; cnx: (null), error: N7Vmacore15SystemExceptionE(Connection refused)
2018-08-08T17:29:41.075Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=LSClient] Caught exception while connecting to LS: N7Vmacore15SystemExceptionE(Connection refused)
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Solution user set to: vpxd-ec2ad075-6aed-89cb-frd5-95b89dfe0140@vcenter.local
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] VC's ServiceId in LookupService:
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] STS URI set to: https://mlxpsc1.corp.com/sts/STSService/vcenter.local
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Admin URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Groupcheck URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] VC SSL certificate location: /etc/vmware-vpx/ssl/rui.crt
2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] STS URI set to: https://mlxpsc1.corp.com/sts/STSService/vcenter.local
2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] Admin URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local
2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] Groupcheck URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local
2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.
2018-08-08T17:29:41.086Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2018-08-08T17:29:41.087Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)
2018-08-08T17:29:41.087Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Will attempt to connect again in 10 seconds.
2018-08-08T17:29:51.087Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.
2018-08-08T17:29:51.097Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2018-08-08T17:29:51.097Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)
.
.
.## A bunch of retries with the exact same output ###
.
.
2018-08-08T17:31:01.186Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Will attempt to connect again in 10 seconds.
2018-08-08T17:31:11.186Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.
2018-08-08T17:31:11.198Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3
2018-08-08T17:31:11.199Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)
2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Max attempts (10) reached. Giving up ...
2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Unable to create SSO facade: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found).
2018-08-08T17:31:11.199Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=VpxProfiler] Init [Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)] took 90122 ms
2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=Main] [Init] Init failed: Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)
--> Backtrace:
-->
--> [backtrace begin] product: VMware VirtualCenter, version: 6.0.0, build: build-7462485, tag: vpxd
--> backtrace[00] libvmacore.so[0x003C5FC4]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
--> backtrace[01] libvmacore.so[0x001F0743]: Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
--> backtrace[02] libvmacore.so[0x0019A69D]: Vmacore::Throwable::Throwable(std::string const&)
--> backtrace[03] vpxd[0x00BD0D8E]: Vmomi::Fault::SystemError::Exception::Exception(std::string const&)
--> backtrace[04] vpxd[0x00BCE80A]
--> backtrace[05] vpxd[0x00BBAAD0]
--> backtrace[06] vpxd[0x00AF8E99]
--> backtrace[07] libc.so.6[0x0001EC36]
--> backtrace[08] vpxd[0x00AF88FD]
--> [backtrace end]
-->
2018-08-08T17:31:11.203Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 94019 ms
2018-08-08T17:31:11.204Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Failed to intialize VMware VirtualCenter. Shutting down...
2018-08-08T17:31:11.204Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=SupportMgr] Wrote uptime information
2018-08-08T17:33:11.205Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Forcing shutdown of VMware VirtualCenter now
So I looked up this error string "Vmomi::Fault::SystemError while trying to connect to SSO Admin server: InvalidResponseExceptionE(Invalid response code: 404 Not Found)" and found a VMWare KB2061412 (https://kb.vmware.com/s/article/2061412) that requests to restart the VMWare Secure Token Service by issusing the following commands (in my case I had to do it in the Platform Controller instead of withing the VCSA)
/etc/init.d/vmware-stsd restart
/etc/init.d/vmware-sts-idmd restart
The article asks you to restart just the vxpd service in the VCSA shell by running /etc/init.d/vmware-vpxd restart, but that only gave me back the VCenter homepage and when trying to load the VSphere Web Client page it would display another error, so I decided to restart all services within VCSA shell by running:
service-control --stop --all
service-control --start --all
And after that I was able to sign out loud the chorus from GF Handel's Messiah "Haaaaaallelujah! Haaaaallelujah!" and was able to login and start managing my VMs.
Note: I also followed this VMWare KB2065630 (https://kb.vmware.com/s/article/2065630) where I added the entry <ThreadStackSizeKb>1024</ThreadStackSizeKb> to the vpxd.cfg file right before doing all the steps mentioned above, so not sure if this played a part.
To recap. I had to update DNS, NTP, add the CA I found in the SSO folder in PSC to the TrustedCerts.pem file and enable it under confix.xml (article mentioned before), then adding the <ThreadStackSizeKb>1024</ThreadStackSizeKb> I just mentioned, followed by restarting the VMWare Secure Token service in the PSC and restarting all services in VCSA.
I had taken a look before at the output of /var/log/vmware/vpxd/vpxd.log and noticed that the SSO connection to the PSC was complaining, but concentrated on the SSL certs since they were not being trusted. I honestly think that it would have probably worked if I had just restarted the Secure Token Service and then all services in the VCSA, but I'll never know unless I restore the whole thing again and try to solve this puzzle.
It's working now, so thanks everyone for your guidance with this adventure! Au revoir!