SSH Error for NSX Corfu Client’s certificates renewal

This is something that I’ve already faced on previously and wrote about in this blogpost.
Probably after the numerous amount of SR opened and feedback from customer VMware/Broadcom decided to develop a script that could be used to renew in a easy way all of these certificates that you can see expiring or expired on your NSX environment.
The certificates that I’m talking about are the one reporting Corfu Client in the name:

• • • CCP-Corfu Client certificate
  • • Cluster Manager-Corfu Client
  • • IDPS reporting-Corfu Client
  • • Monitoring-Corfu Client
  • MP-Corfu Client

You can see they are quite a lot because they are basically each one of kind for each manager appliance so, you can understand that even renew them with self signed method is for sure a tedious task.

I got through this interesting KB -> https://knowledge.broadcom.com/external/article/324175/nsx-alarms-indicating-certificates-have.html and seems there is a scirpted way in order to replace automatically them and then forget about forever.

Scripted Resolution

VMware have developed a script that will replace the certificates automatically and resolve this issue.
The validity period of the certificates generated by this script is 100 years for CBM certificates, and 825 days for others.
The script replaces the following certificates API, MGMT_CLUSTER, APH_TN, APH(AR), LOCAL_MANAGER(on LM), GLOBAL_MANAGER(on GM), CBM_CLUSTER_MANAGER, CBM_CORFU and CCP.

• • • The script is compatible with NSX version 4.1.0 and above.
  • • The script can be run on both Federation(GM/LM) and non-Federation environments.
  • • An NSX backup must be taken before running the script. Also, ensure the passphrase is known.
  • • The script will replace the certs that are expired or expiring in the next 31 days. If the certs have longer than 31 days of validation, it is possible to change LEAD_DAYS in the script to consider them.
  • • This is a python version 3 script which must be run from a client machine which has paramiko, cryptography and pyopenssl python packages installed.
  • • Depending on the system, these packages may be installed with a command such as:
      # sudo pip3 install cryptography
  • • These packages are already installed on VCSA (vCenter Server Appliance), hence this can be used as a client machine to execute the script.
  • • The script cannot be run directly on the NSX Manager, as it does not have the required python modules. It is not supported to install it on the NSX Manager.
  • • The script is also supported to be run from a Windows machine.
  • • Communication to the NSX Manager VIP/IP on port TCP 443 (HTTPS) is required.
  • Communication to the NSX Manager VIP/IP on port TCP 22 (SSH) is also required if running NSX 4.1.1.

In my case what I’ve used the suggested way so upload the script to the VCSA that is linked to the NSX environment and launch from there the script.

My usual bad luck, I faced an issue when launching the python script from the VCSA vs NSX Managers: SshCommandExecutor: An error occurred: [digital envelope routines] unsupported

THE ISSUE
seems the OpenSSH version for this VMware VirtualCenter 8.0.3 build-24091160 is the one causing this connection error and blocking the script before the real run on the nsx nodes. At the moment I didn’t find out which is the big differences but a PR on Engineering was opened and once I’ve an update I’ll wrote down here.

THE SOLUTION:
I tried with a different vCenter version 7.0.3 and the script worked smoothly without any issue, connecting to each node and being able to renew the certificates automatically with expiration date in 100 years.

The other possibility is to build up a Windows or Linux client machine with the python packets written as requirement in the KB:

• paramiko, cryptography and pyopenssl python packages installed.
• Depending on the system, these packages may be installed with a command such as:
  # sudo pip3 install cryptography