Table of Contents

Summary

In this process we will be upgrading vCenter server 7.0 Update 3d (U3d) to the latest vCenter server 7.0 Update 3o (U3o). This update includes the out-of-bounds patches (VMSA-2023-0023) to ensure that vulnerability is not present within your environment. This procedure is written assuming the vCenter is 100% network isolated and does not have access to directly download from VMware.com to apply updates. My average upgrade time is around one hour however some are taking as long as three hours.

VMware Documentation

vCenter 7.0 Update 3d release notes.

Prerequisites

  1. Confirm the existing vCenter is working as expected and there are no errors or issues.
  2. All hypervisors have to be running ESXi 6.5 or newer.
  3. All VDS need to be at least 6.5.0. You can get the versions of DVS by running the command ‘Get-VDSwitch’ via PowerCLI.
  4. Make sure all of your hypervisors are connected to the vCenter. Any disconnected hypervisors or disabled hypervisors should be addressed prior to starting to ensure best possible outcome.
  5. Create the following as part of the backout/rollback plan:
    1. Create a manual backup of your vCenter through the VAMI.
    2. Create a clone of vCenter VM.
    3. Create a snapshot of the vCenter VM.
  6. Ensure you have plenty of storage on the vCenter virtual appliance by logging into the shell and running the commands ‘df -h’ and looking at the output. You can also run ‘du -x -d1 -h’ which can tell you where there is large space being consumed at. You can also sort files via total file size by using ‘ls -Shal’. Frequently you will find lots of files that can be purged in order to ensure you have plenty of space.
  7. If your root password of the appliance has not been recently changed then log in and confirm it has not expired. If it has then change it prior to starting the upgrade process. To do this log into the vCenter Appliance webpage at https://IP-Address:5480
  8. vCenter License - will NOT need to be upgraded.
  9. Download locations:
    1. vCenter 7.0 Update 3d
    2. Look for ‘VC’ and select ‘7.0.3’ and click ‘search’.
    3. Under VC-7.0u3o download ‘VMware-vCenter-Server-Appliance-7.0.3.01700-22357613-patch-FP.iso’

Upgrade vCenter 7.0 to vCenter 7.0U3d

  1. Download the upgrade-script.iso, located here, to the jump box and mount it to the vCenter appliance. This is just an iso I created from the KB script located in VMware KB83145.
  2. OPTINAL: Log into the VAMI, click on services, and scroll down to ‘vSan Health Service’ and set it to manual startup type and stop the service.
  3. Log into the vCenter via SSH, gain shell access and mount the upgrade-script CD ROM.
    shell
    mount /dev/cdrom /mnt/cdrom
    
  4. Copy the upgrade script to the /tmp/ directory.
    cp /mnt/cdrom/KB_83145_PatchCleanupScript.sh /tmp/
    
  5. Change the directory to /tmp/
    cd /tmp/
    
  6. Change the file to be executable.
    chmod +x KB_83145_PatchCleanupScript.sh
    
  7. Run the script and press ‘Y’ when asked.
    ./KB_83145_PatchCleanupScript.sh
    
  8. Clean up the vSAN service historical health information database tables. This allows services to start quicker allowing for highest chance of success on service-startup timeouts.
    /usr/bin/dbcc -fbss embedded
    
  9. Stop the applmgmt, vmdird, and vmware-updatemgr services.
    service-control --stop applmgmt vmdird vmware-updatemgr
    
  10. Reset the update manager database and delete the old patch files. This allows the update manager service to start quicker allowing for highest chance of success on service-startup timeouts.
    /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db
    rm -rf /storage/updatemgr/patch-store/*
    
  11. Run commands for vmdir.password and vmdir.username.
    /bin/install-parameter -s vmdir.password
    /bin/install-parameter -s vmdir.username
    
  12. Purge any old software updates existing on the appliance.
    rm -rf /storage/core/software-update/updates/*
    rm -rf /storage/core/software-update/stage/*
    
  13. Relocate the software_update_state.conf file.
    mv /etc/applmgmt/appliance/software_update_state.conf /storage/core
    
  14. Configure the database timeout. This allows for longer time cycle in order for the database changes to take place.
    cd /
    /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\vmdir\Parameters]' "CopyDbIntervalInSec" "0"
    
  15. Start the services back up.
    service-control --start applmgmt vmdird vmware-updatemgr
    
  16. Attach the VMware-vCenter-Server-Appliance-7.0.3.01700-22357613-patch-FP.iso file to the vCenter appliance CD-ROM.
  17. Log into the VAMI and click on ‘update’ and wait for it to display information. If nothing appears click on the ‘Check updates’ and select ‘Check CD ROM’. Ensure 7.0.3.01700 is selected and click on the ‘Stage and Install’. vCenter Server 7.0 U3o Image 1
  18. Accept the EULA and select ‘next’. vCenter Server 7.0 U3o Image 2
  19. Allow the prechecks to run and complete. vCenter Server 7.0 U3o Image 3
  20. If asked join the CEIP program and select ‘next’.
  21. Ensure you have backed up the vCenter earlier and select that you have, then click finish. vCenter Server 7.0 U3o Image 5
  22. Allow the process to proceed. The vCenter will reboot at the very end of the process after which you can log back into the VAMI and vCenter. vCenter Server 7.0 U3o Image vCenter Server 7.0 U3o Image vCenter Server 7.0 U3o Image vCenter Server 7.0 U3o Image vCenter Server 7.0 U3o Image vCenter Server 7.0 U3o Image

    NOTE: If the conversion of data fails to complete in time then the update will typically spawn a ‘resume’ button. If it does then click the button to allow the process to proceed normally. vCenter Server 7.0 U3o Image

  23. If you disabled the vSan Health Service, then log back into the VAMI and locate the ‘vSan Health Service’ and start the service, and configure it to automatic startup type.

Backout/Rollback Plan

  1. Revert to the snapshot created at the first upgrade step. Confirm the snapshot allowed the configuration to come back online, if it does not then proceed to next step.
  2. Restore the clone that was created in the second upgrade step. Confirm the clone allowed the configuration to come back online, if it does not then proceed to next step.
  3. Create a new vCenter appliance and restore backups created in VAMI interface. Confirm this step allows the vCenter to come back online, if it does not then proceed to next step.
  4. Contact VMware customer service by creating a new Service Request.