Did you replace your vCenter certificate from a custom one or are you using VMCA certificates?

Thanks for replying. Update so far: I followed this article Dude! Where's my vCSA SSL Cert chain? - vRyan.co.uk - Virtualization Blog  from someone experiencing this certificate issue, and like theVElement mentions, the trustpoints.pem file in the rhttpproxy SSL container was empty and config.xml had it commented on both the VCSA and the PSC appliances, which I don't know how it was working before the restore since it's in the rhttpproxy config from the backup I had.

However the problem still remains even after a reboot of both devices. Now this whole time I think the problem has been that the VPXD service in the VCSA never listens on port TCP 8089 as seen in my manual restart attempt of vmware-vpxd. Looking at logs from months before the restore, the messsage "vmware-vpxd: VC SSL Certificate does not exist, it will be generated by vpxd" has always been there and the VPXD used to start and listen on 8089 after a few seconds.

if I open the shell of VCSA and type "iptables -L port_filter -n --line-numbers" the port 8089 is never listed as listening, so I don't think it's a firewall issue here. The output is not very helpful as to what is going on.

mgmlxvcs1:~ # /etc/init.d/vmware-vpxd restart

vmware-vpxd: already stopped

vmware-vpxd: VC SSL Certificate does not exist, it will be generated by vpxd

Waiting for the embedded database to start up: success

Executing pre-startup scripts...

vmware-vpxd: Starting vpxd by administrative request.

success

vmware-vpxd: Waiting for vpxd to start listening for requests on 8089

Waiting for vpxd to initialize: ..........................................................Wed Aug  8 00:26:27 UTC 2018 Captured live core: /var/core/live_core.vpxd.6497.08-08-2018-00-26-27

[INFO] writing vpxd process dump retry:2 Time(Y-M-D H:M:S):2018-08-08 00:26:25

.Wed Aug  8 00:26:39 UTC 2018 Captured live core: /var/core/live_core.vpxd.6497.08-08-2018-00-26-39

[INFO] writing vpxd process dump retry:1 Time(Y-M-D H:M:S):2018-08-08 00:26:37

.failed

failed

vmware-vpxd: vpxd failed to initialize in time.

vpxd is already starting up. Aborting the request.

Is this vSphere 6.0 or 6.5?

You'll want to look in /var/log/vmware/vpxd/vpxd.log to see if there's any indication why vCenter is taking so long to start.

root@XXXXX[ ~ ]# service-control --start vmware-sts-idmd

Perform start operation. vmon_profile=None, svc_names=['vmware-sts-idmd'], include_coreossvcs=False, include_leafossvcs=False

2018-08-08T22:46:01.701Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmware-sts-idmd']

2018-08-08T22:46:01.718Z   Done running command

2018-08-08T22:46:01.726Z   Service vmware-sts-idmd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

2018-08-08T22:46:01.726Z   Running command: ['/sbin/service', u'vmware-sts-idmd', 'status']

2018-08-08T22:46:01.785Z   Done running command

2018-08-08T22:46:01.794Z   Running command: ['/usr/bin/systemctl', 'daemon-reload']

2018-08-08T22:46:01.956Z   Done running command

2018-08-08T22:46:01.956Z   Running command: ['/usr/bin/systemctl', 'set-property', u'vmware-sts-idmd.service', 'MemoryAccounting=true', 'CPUAccounting=true', 'BlockIOAccounting=true']

2018-08-08T22:46:01.984Z   Done running command

2018-08-08T22:46:03.226Z   RC = 1

Stdout =

Stderr = Job for vmware-sts-idmd.service failed because the control process exited with error code. See "systemctl status vmware-sts-idmd.service" and "journalctl -xe" for details.

2018-08-08T22:46:03.226Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Stderr: Job for vmware-sts-idmd.service failed because the control process exited with error code. See \"systemctl status vmware-sts-idmd.service\" and \"journalctl -xe\" for details.\n"

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Stderr: Job for vmware-sts-idmd.service failed because the control process exited with error code. See \"systemctl status vmware-sts-idmd.service\" and \"journalctl -xe\" for details.\n'",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Error executing start on service vmware-sts-idmd. Details {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-sts-idmd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-sts-idmd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Service-control failed. Error {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-sts-idmd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-sts-idmd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

please if you can help me I need to solve this issue since not all the services of the external psc upload

Message was edited by: Jose Franco

It's running 6.0

Alright it's back up! Success! After taking a closer look in the VCSA's /var/log/vmware/vpxd/vpxd.log line by line after the SSL certs are exchanged between VCSA and the PSC, the VCSA was complaining that it could not find the SSO Admin server (/sso-adminserver/sdk/vcenter.local) when connecting to the PSC with 404 not found error as seen below:

2018-08-08T17:29:41.070Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=dbPortgroup] [VpxdInvtDVPortGroup::PreLoadDvpgConfig] loaded [9] dvpg config objects

2018-08-08T17:29:41.074Z warning vpxd[7F5E84185700] [Originator@6876 sub=Default] Failed to connect socket; <io_obj p:0x00007f5e80e73f10, h:25, <TCP '0.0.0.0:0'>, <TCP '127.0.0.1:18090'>>, e: system:111(Connection refused)

2018-08-08T17:29:41.075Z error vpxd[7F5E84185700] [Originator@6876 sub=HttpConnectionPool-000001] [ConnectComplete] Connect failed to <cs p:00007f5e80f162e0, TCP:localhost:18090>; cnx: (null), error: N7Vmacore15SystemExceptionE(Connection refused)

2018-08-08T17:29:41.075Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=LSClient] Caught exception while connecting to LS: N7Vmacore15SystemExceptionE(Connection refused)

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Solution user set to: vpxd-ec2ad075-6aed-89cb-frd5-95b89dfe0140@vcenter.local

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] VC's ServiceId in LookupService:

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] STS URI set to: https://mlxpsc1.corp.com/sts/STSService/vcenter.local

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Admin URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Groupcheck URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] VC SSL certificate location: /etc/vmware-vpx/ssl/rui.crt

2018-08-08T17:29:41.077Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] STS URI set to: https://mlxpsc1.corp.com/sts/STSService/vcenter.local

2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] Admin URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local

2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][CreateSsoFacade]] [CreateUserDirectory] Groupcheck URI set to: https://mlxpsc1.corp.com/sso-adminserver/sdk/vcenter.local

2018-08-08T17:29:41.078Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.

2018-08-08T17:29:41.086Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-08-08T17:29:41.087Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)

2018-08-08T17:29:41.087Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Will attempt to connect again in 10 seconds.

2018-08-08T17:29:51.087Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.

2018-08-08T17:29:51.097Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-08-08T17:29:51.097Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)

.

.

.## A bunch of retries with the exact same output ###

.

.

2018-08-08T17:31:01.186Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Will attempt to connect again in 10 seconds.

2018-08-08T17:31:11.186Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [CreateServiceContent] Try to connect to SSO VMOMI endpoint.

2018-08-08T17:31:11.198Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Closing Response processing in unexpected state: 3

2018-08-08T17:31:11.199Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Vmomi::Fault::SystemError while trying to connect to SSO Admin server: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found)

2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoCertificateManagerImpl]] [RetryOnConnectionFailure] Max attempts (10) reached. Giving up ...

2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=[SSO][SsoFactory_CreateFacade]] Unable to create SSO facade: N7Vmacore4Soap24InvalidResponseExceptionE(Invalid response code: 404 Not Found).

2018-08-08T17:31:11.199Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=VpxProfiler] Init [Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)] took 90122 ms

2018-08-08T17:31:11.199Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=Main] [Init] Init failed: Vpx::Common::Sso::SsoFactory_CreateFacade(sslContext, ssoFacadeConstPtr)

--> Backtrace:

-->

--> [backtrace begin] product: VMware VirtualCenter, version: 6.0.0, build: build-7462485, tag: vpxd

--> backtrace[00] libvmacore.so[0x003C5FC4]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)

--> backtrace[01] libvmacore.so[0x001F0743]: Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)

--> backtrace[02] libvmacore.so[0x0019A69D]: Vmacore::Throwable::Throwable(std::string const&)

--> backtrace[03] vpxd[0x00BD0D8E]: Vmomi::Fault::SystemError::Exception::Exception(std::string const&)

--> backtrace[04] vpxd[0x00BCE80A]

--> backtrace[05] vpxd[0x00BBAAD0]

--> backtrace[06] vpxd[0x00AF8E99]

--> backtrace[07] libc.so.6[0x0001EC36]

--> backtrace[08] vpxd[0x00AF88FD]

--> [backtrace end]

-->

2018-08-08T17:31:11.203Z warning vpxd[7F5E96A7F7A0] [Originator@6876 sub=VpxProfiler] ServerApp::Init [TotalTime] took 94019 ms

2018-08-08T17:31:11.204Z error vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Failed to intialize VMware VirtualCenter. Shutting down...

2018-08-08T17:31:11.204Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=SupportMgr] Wrote uptime information

2018-08-08T17:33:11.205Z info vpxd[7F5E96A7F7A0] [Originator@6876 sub=Default] Forcing shutdown of VMware VirtualCenter now

So I looked up this error string "Vmomi::Fault::SystemError while trying to connect to SSO Admin server: InvalidResponseExceptionE(Invalid response code: 404 Not Found)" and found a VMWare KB2061412 (https://kb.vmware.com/s/article/2061412) that requests to restart the VMWare Secure Token Service by issusing the following commands (in my case I had to do it in the Platform Controller instead of withing the VCSA)

/etc/init.d/vmware-stsd restart
/etc/init.d/vmware-sts-idmd restart

The article asks you to restart just the vxpd service in the VCSA shell by running /etc/init.d/vmware-vpxd restart, but that only gave me back the VCenter homepage and when trying to load the VSphere Web Client page it would display another error, so I decided to restart all services within VCSA shell by running:

service-control --stop --all

service-control --start --all

And after that I was able to sign out loud the chorus from GF Handel's Messiah "Haaaaaallelujah! Haaaaallelujah!" and was able to login and start managing my VMs.

Note: I also followed this VMWare KB2065630 (https://kb.vmware.com/s/article/2065630) where I added the entry <ThreadStackSizeKb>1024</ThreadStackSizeKb> to the vpxd.cfg file right before doing all the steps mentioned above, so not sure if this played a part.

To recap. I had to update DNS, NTP, add the CA I found in the SSO folder in PSC to the TrustedCerts.pem file and enable it under confix.xml (article mentioned before), then adding the <ThreadStackSizeKb>1024</ThreadStackSizeKb> I just mentioned, followed by restarting the VMWare Secure Token service in the PSC and restarting all services in VCSA.

I had taken a look before at the output of /var/log/vmware/vpxd/vpxd.log and noticed that the SSO connection to the PSC was complaining, but concentrated on the SSL certs since they were not being trusted. I honestly think that it would have probably worked if I had just restarted the Secure Token Service and then all services in the VCSA, but I'll never know unless I restore the whole thing again and try to solve this puzzle.

It's working now, so thanks everyone for your guidance with this adventure! Au revoir!

Oh yeah, forgot that I also started all Platform controller VMWare services manually using:

service-control --start vmafdd

service-control --start vmware-rhttpproxy

service-control --start vmdird

service-control --start vmcad

service-control --start vmware-stsd

service-control --start vmware-cm

service-control --start vmware-cis-license

service-control --start vmware-psc-client

service-control --start vmware-sca

service-control --start applmgmt

service-control --start vmware-syslog

service-control --start vmware-syslog-health

and I just noticed that I'm missing in this list the second service mentioned in the article KB2061412 which is vmware-sts-idmd and it would have probably worked a long time ago. Oh well!

Can you check if there is a process already running?

ps aux | grep -i idmd

If yes, kill the process and try to start the service.

Please consider marking this answer as "correct" or "helpful" if you think your questions have been answered.

Cheers,

Supreet

change the Bit length of the certificate to 2048 from any higher value. The issue should be fixed. vCenter is detecting the heartbeat coming from the Host as an insecure heartbeat.

Guru

I have inherited a vCenter 5.5 server running on Windows 2008R2.  My predecessor installed some kind of a script, I either powershell or maybe perl, that allows it to communicate with our support ticketing system, such that a ticket is opened automatically any time there is an alarm on the system.  The problem is the script wasn't finished, so we get tickets on *any* change of alarm state, including green and grey alarms which really don't warrant any action.  Needless to say our callout people are getting more than a little tired of this thing paging them at all hours of day and night.  I have looked through the alarm definitions and there are no actions to run a script attached to any of them, yet it is creating tickets for all of them.  I can't even find where the script file lives on the server.

Can anyone suggest how he might have done this, and how I can remove it?  It doesn't seem to be polled, it's definitely event-driven.  An example of the information it sends follows.  Thanks very much.

Metric:
alarm.HealthStatusChangedAlarm - Event: Status change (9943329) Summary:
[FQDN of server] (vCenter) status changed from red to green
Date: 09/08/2018 03:48:55 User name: Health Service Arguments: componentId =
E871136A-6708-4DE3-8E8B-6E4E560A93D2 componentName =
[FQDN of server] (vCenter) newStatus = green oldStatus = red |
NewStatus: Gray | ObjectName: Datacenters | ObjectType: other | OldStatus: Gray

after executing the command I get the following information. it seems that he was not running

root@SAPL1454 [ ~ ]# ps aux | grep -i idmd

root       1904  0.0  0.0  10592    24 ?        Ss   Aug08   0:00 vmware-sts-idmd -procname vmware-sts-idmd -wait 120 -server -Xmx179m -XX:CompressedClassSpaceSize=160m -Xss256k -XX:ParallelGCThreads=1 -XX:ErrorFile=/var/log/vmware/sso/hs_err_idm_pid%p.log -XX:HeapDumpPath=/var/log/vmware/sso -home /usr/java/jre-vmware -pidfile /var/run/vmware-sts-idmd.pid -errfile /var/log/vmware/sso/vmware-sts-idmd.err -cp /opt/vmware/lib64/commons-lang-2.5.jar:/opt/vmware/lib64/commons-logging-1.1.1.jar:/opt/vmware/lib64/log4j-api-2.2.jar:/opt/vmware/lib64/log4j-1.2-api-2.0.2.jar:/opt/vmware/lib64/log4j-core-2.2.jar:/opt/vmware/lib64/log4j-1.2.12rsa-1.jar:/opt/vmware/lib64/jna.jar:/opt/vmware/lib64/commons-daemon-1.0.8.jar:/opt/vmware/lib64/bcpkix-jdk15on-147.jar:/opt/vmware/lib64/bcprov-jdk15on-147.jar:/opt/vmware/lib64/vmware-identity-platform.jar:/opt/vmware/lib64/vmware-identity-idm-interface.jar:/opt/vmware/lib64/vmware-identity-idm-server.jar:/opt/vmware/lib64/vmware-identity-diagnostics.jar:/opt/vmware/lib64/bcprov-jdk16-145.jar:/opt/vmware/lib64/authentication-framework.jar:/opt/vmware/lib64/afd-heartbeat-service.jar:/opt/vmware/lib64/httpclient-4.5.1.jar:/opt/vmware/lib64/authapi-8.5.jar:/opt/vmware/lib64/cryptoj-6.1.jar:/opt/vmware/lib64/xmlsec-1.5.5.jar:/opt/vmware/lib64 -Djava.rmi.server.codebase=file:////opt/vmware/lib64/vmware-identity-idm-interface.jar file:////opt/vmware/lib64/vmware-identity-idm-server.jar -Djava.security.policy=/opt/vmware/share/config/server.policy -Dvmware.log.dir=/var/log/vmware/sso -Dlog4j.configurationFile=file:///opt/vmware/share/config/log4j2.xml com.vmware.identity.idm.server.IdmServer

root       1905  1.2  5.2 1922520 210956 ?      Sl   Aug08  16:26 vmware-sts-idmd -procname vmware-sts-idmd -wait 120 -server -Xmx179m -XX:CompressedClassSpaceSize=160m -Xss256k -XX:ParallelGCThreads=1 -XX:ErrorFile=/var/log/vmware/sso/hs_err_idm_pid%p.log -XX:HeapDumpPath=/var/log/vmware/sso -home /usr/java/jre-vmware -pidfile /var/run/vmware-sts-idmd.pid -errfile /var/log/vmware/sso/vmware-sts-idmd.err -cp /opt/vmware/lib64/commons-lang-2.5.jar:/opt/vmware/lib64/commons-logging-1.1.1.jar:/opt/vmware/lib64/log4j-api-2.2.jar:/opt/vmware/lib64/log4j-1.2-api-2.0.2.jar:/opt/vmware/lib64/log4j-core-2.2.jar:/opt/vmware/lib64/log4j-1.2.12rsa-1.jar:/opt/vmware/lib64/jna.jar:/opt/vmware/lib64/commons-daemon-1.0.8.jar:/opt/vmware/lib64/bcpkix-jdk15on-147.jar:/opt/vmware/lib64/bcprov-jdk15on-147.jar:/opt/vmware/lib64/vmware-identity-platform.jar:/opt/vmware/lib64/vmware-identity-idm-interface.jar:/opt/vmware/lib64/vmware-identity-idm-server.jar:/opt/vmware/lib64/vmware-identity-diagnostics.jar:/opt/vmware/lib64/bcprov-jdk16-145.jar:/opt/vmware/lib64/authentication-framework.jar:/opt/vmware/lib64/afd-heartbeat-service.jar:/opt/vmware/lib64/httpclient-4.5.1.jar:/opt/vmware/lib64/authapi-8.5.jar:/opt/vmware/lib64/cryptoj-6.1.jar:/opt/vmware/lib64/xmlsec-1.5.5.jar:/opt/vmware/lib64 -Djava.rmi.server.codebase=file:////opt/vmware/lib64/vmware-identity-idm-interface.jar file:////opt/vmware/lib64/vmware-identity-idm-server.jar -Djava.security.policy=/opt/vmware/share/config/server.policy -Dvmware.log.dir=/var/log/vmware/sso -Dlog4j.configurationFile=file:///opt/vmware/share/config/log4j2.xml com.vmware.identity.idm.server.IdmServer

root      77922  0.0  0.0  13876  2272 pts/0    S+   09:38   0:00 grep --color=auto -i idmd

1) Take a snapshot of the VM.

2) Run the command <kill 1904>.

3) Run the command <ps aux | grep -i idmd> to validate if both processes 1904 and 1905 have been killed. If 1905 still shows up, run <kill 1905> to kill it.

4) Run the command <service-control --stop vmware-sts-idmd>.

5) Now, run the command <service-control --start vmware-sts-idmd>.

Please consider marking this answer as "correct" or "helpful" if you think your questions have been answered.

Cheers,

Supreet

It seems that it did not work. My son's mistake is to throw when I try to lift the service. I followed the steps you indicated but still nothing

root@SAPL1454 [ ~ ]# kill 1904

root@SAPL1454 [ ~ ]# ps aux | grep -i idmd

root     108905  0.0  0.0  13872  2344 pts/0    S+   11:14   0:00 grep --color=auto -i idmd

root@SAPL1454 [ ~ ]# service-control --stop vmware-sts-idmd

Perform stop operation. vmon_profile=None, svc_names=['vmware-sts-idmd'], include_coreossvcs=False, include_leafossvcs=False

2018-08-09T15:15:36.450Z   Service vmware-sts-idmd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

2018-08-09T15:15:36.450Z   Running command: ['/sbin/service', u'vmware-sts-idmd', 'status']

2018-08-09T15:16:10.406Z   Done running command

Successfully stopped service vmware-sts-idmd

root@SAPL1454 [ ~ ]# service-control --start vmware-sts-idmd

Perform start operation. vmon_profile=None, svc_names=['vmware-sts-idmd'], include_coreossvcs=False, include_leafossvcs=False

2018-08-09T15:16:15.105Z   Running command: ['/usr/bin/systemctl', 'is-enabled', u'vmware-sts-idmd']

2018-08-09T15:16:17.386Z   Done running command

2018-08-09T15:16:17.395Z   Service vmware-sts-idmd does not seem to be registered with vMon. If this is unexpected please make sure your service config is a valid json. Also check vmon logs for warnings.

2018-08-09T15:16:17.395Z   Running command: ['/sbin/service', u'vmware-sts-idmd', 'status']

2018-08-09T15:16:17.455Z   Done running command

2018-08-09T15:16:17.465Z   Running command: ['/usr/bin/systemctl', 'daemon-reload']

2018-08-09T15:16:42.486Z   Done running command

2018-08-09T15:16:42.487Z   Invoked command: ['/usr/bin/systemctl', 'daemon-reload']

2018-08-09T15:16:42.487Z   RC = 1

Stdout =

Stderr = Failed to execute operation: Connection timed out

2018-08-09T15:16:42.487Z   {

    "resolution": null,

    "detail": [

        {

            "args": [

                "Command: ['/usr/bin/systemctl', 'daemon-reload']\nStderr: Failed to execute operation: Connection timed out\n"

            ],

            "id": "install.ciscommon.command.errinvoke",

            "localized": "An error occurred while invoking external command : 'Command: ['/usr/bin/systemctl', 'daemon-reload']\nStderr: Failed to execute operation: Connection timed out\n'",

            "translatable": "An error occurred while invoking external command : '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Error executing start on service vmware-sts-idmd. Details {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-sts-idmd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-sts-idmd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

Service-control failed. Error {

    "resolution": null,

    "detail": [

        {

            "args": [

                "vmware-sts-idmd"

            ],

            "id": "install.ciscommon.service.failstart",

            "localized": "An error occurred while starting service 'vmware-sts-idmd'",

            "translatable": "An error occurred while starting service '%(0)s'"

        }

    ],

    "componentKey": null,

    "problemId": null

}

thanks, it was very helpful

I called a VM ole'windows2012r2 in vCenter 6.7.  When taking and removing snapshots, the Recent Tasks pane incorrectly displays the x27 character code.

Even though it displays correctly in the Hosts & Clusters view here:

If anyone from support sees this, is there already an engineering report for this?  I know it's small, but kinda silly that parts of the UI render differently and should be an easy fix to correctly display the characters.

I think it's a generic issue encoding the characters, since a " also does something similar:

vCenter is build 8546293

ESXi is build 8941472

From tasks/event I see

From Hosts and clusters tree, I see:

KB 2046088 use to be around for special characters in VM names ... but it's no longer published.

VMware recommend you avoid using special charters within VM names and datastores.

Yeah I've notice that as well, most of the articles about special charters seem to have.

Where is that referenced?  I've been trying to find supporting documentation around that, but the last KB I saw published is no longer available from kb.vmware.com.

Also, it's just a display name ... and I know they use it in the pathing, but the VM originated as windows2012r2 ... and has not be SvMotioned to re-create the DIRs.

I personally think it's kind of silly to be this resistant to any special character in some places but not others.  If it wasn't allowed, then don't let me rename the VM to include offending characters.

Agreed. This might be likely the cause.

In addition, if you migrate both CPU and Storage on the next screen it is showing info about attached devices..

This way you can edit the VM hardware settings accordingly. For example by choosing Client Device to CD/DVD drive

************************************

*** This is the actual Answer ***

************************************

OK I actually got to the root of the problem by calling VMWare support since my search for PSC services not starting on auto was not yielding any results on my own. After explaining the issue and trying to prove that the problem is that the PSC services are currently set to Manual and that I just needed the right commands on how to set them to Auto from the shell, they asked me to follow KB2151528 (https://kb.vmware.com/s/article/2151528#/s/article/2151528) which says the following steps

1. Run this command to check the start status : chkconfig -A

For example:

#chkconfig -A

You see output similar to:

applmgmt off

vmafdd off
vmcad off
vmci on
vmdird off
vmware-cis-license off
vmware-cm off
vmware-psc-client off
vmware-rhttpproxy off
vmware-sca off
vmware-sts-idmd off
vmware-stsd off
vmware-syslog off
vmware-syslog-health off
vmware-tools-services on
vmware-tools-vgauth on
vsock on

In the above example, most of the VMware service status is OFF.

2. If the service status is set to off, then perform these steps to turn it ON.

a. Run this command to stop all the running services:

service-control --stop --all

b. Run this command to change the service state from Off to On state:

chkconfig <servicename> on

For example:
chkconfig vmware-cm on

c. Once all the service status is set to ON, run this command to start all the services:

service-control --start --all

Note: Change the status of the dependent service to ON before changing the status of any service that you are trying to start. Attempting to change the status without changing the status of the dependent service will fail.

For example:

#chkconfig vmware-cis-license on
insserv: FATAL: service vmware-cm has to be enabled to use service vmware-cis-license
insserv: Service syslog is missed in the runlevels 2 to use service cgrulesengd
insserv: exiting now!
/sbin/insserv failed, exit code 1

So I used my Excell chops and created this list and pasted it a few times in the shell of the PSC until I got no errors

chkconfig  applmgmt on

chkconfig  vmafdd on

chkconfig  vmcad on

chkconfig  vmdird on

chkconfig  vmware-cis-license on

chkconfig  vmware-cm on

chkconfig  vmware-psc-client on

chkconfig  vmware-rhttpproxy on

chkconfig  vmware-sca on

chkconfig  vmware-sts-idmd on

chkconfig  vmware-stsd on

chkconfig  vmware-syslog on

chkconfig  vmware-syslog-health on

And Voila! After a restart of both PSC and VCSA all is well again!