Can you try not to mix IP and DNS names? Certificates in a vSphere environment are pretty allergic to that. Not to even mention the hard requirement of forward and reverse DNS for SSO to actually be supported. As all your service endpoints are registered by FQDN use the FQDN in your commands as well 
What happens when you try to reset the admin@system-domain password using the following kb?
VMware KB: Unlocking and resetting the vCenter Single Sign On (SSO) administrator password