Hello All
So I ended up solving the issue myself. After a few hours of sleep and a little more banging of the head against the table I need to take the blame on myself for not reading and checking every little last bit of advice.
VMware explicitly states in KB 2057223 to OpenSSL v0.9.8. If you are on any recent MacOS or Linux you are most probably using OpenSSL v1.0.1 (you may check by issuing 'openssl version' on the CLI). The reason behind that is stated a few lines below the requirement of OpenSSL 0.9.8 but just not pointed out clearly: OpenSSL >1.0 uses per default PKCS#8 format for private keys. But a few services of vCenter just do not support that. Which ones you figured out already since you are reading this thread 
So how to resolve this issue you ask? Easy: the very same KB helps you there as well. Just issue this command to convert you private key:
openssl rsa -in pk8.key -out pk1.key
After that just follow the guidelines again to replace the certificate of the vCenter service (steps 1-12), give the appliance a reboot and you should be all set.
To be on the safe side you should convert all private key and run thru the whole guideline to replace all the certificates again.
Hope this helps.
Cheers
Roman