Upgrade vCloud Director 9.5 to 10.1.1

2020, Sep 20    

Table of Contents

Overview

Our configuration at the start of this process is vCloud Director 9.5 installed and successfully running on CentOS7 VM using an external MS SQL database. By the end of this process we will be running Cloud Director on the self-contained appliance with a single external NFS share. I highly advise you create a cloned environment and run through these steps to ensure they work correctly in your environment as this is a highly complex process. If you run multiple cells then I suggest you perform this work on a single cell, upgrading it to the very end, remove all additional cells and simply deploy new cells once the primary is successfully working. This method assumes isolation from direct access to download updates, due to this the article is written around creating ISOs, mounting CDs, etc. With Cloud Director 10 the old flex UI has been completely deprecated. There are many new items in Cloud Director 10.1 which I have summarized in my article vCloud Director 10.1 Release.

Create an original restore point.

  1. Create a snapshot of your vCloud Director VM. If possible I would create a clone as well.
  2. Create a backup for your SQL vCloud database. If possible I would create a snapshot and clone as well.

Upgrade existing Cloud Director from 9.5 to 9.7

  1. Download vCloud Director bin file “VMware vCloud Director 9.7.0.4-Binary for Linux Installation or Upgrade” from VMware.
  2. Using an iso tool create an .iso of the .bin file. VCD97-linux.iso I personally like using Folder2ISO that is available for Windows.
  3. Log into your existing Linux VM via SSH.
  4. Create a new director for mounting the CDROM. 
    mkdir /mnt/cdrom
  5. Mount the CDRom so you can utilize the new .bin file. 
    mount /dev/cdrom /mnt/cdrom
  6. Copy the file to root. 
    cp /mnt/cdrom/vmware-vcloud-director-distribution-9.7.0-15498290.bin /
  7. Change director to the upload location 
    cd /
  8. Change the file to executable: 
    chmod u+x vmware-vcloud-director-distribution-9.7.0-15498290.bin
  9. Check the cell status to ensure no jobs are running. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
  10. Quiesce the cell to prevent new jobs from running and complete existing jobs. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell -q true
  11. Shutdown vCloud Director management cell. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
/opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –shutdown
  12. Update vCloud Director from 9.5 to 9.7 by executing the .bin file. 
    ./vmware-vcloud-director-distribution-9.7.0-15498290.bin
  13. The installer starts and then prompts you if you wish to continue the upgrade, select: Y
  14. Allow the installer to complete fully.
  15. Upgrade the database by running the upgrade command. 
    /opt/vmware/vcloud-director/bin/upgrade
    1. The upgrade DB process starts and then prompts you “Do you wish to upgrade the product now?”, answer with “Y” to continue.
    2. The process will continue and generally you will see the upgrade may not be safe for all cells, answer with “Y” to continue.
    3. You will be prompted to backup your database, if you did not do it earlier then do it now, then press “Y” to continue.
    4. When asked if you want to start now enter Y unless you have additional cells you need to upgrade.

      NOTE If you need to manually start vCloud Director services manually run the following command. service vmware-vcd start

  16. Once vCloud Director is back online you will want to confirm that everything has worked correctly to this point, your VCD is populated, showing 9.7, etc.

Re-Using the existing vCloud Director IPs.

If you plan on keeping the same IP for vCloud Director then we will need to change the existing IP for the current primary. If you run multiple cells then I suggest you wait to deploy them until you get your first cell completely upgraded and then deploy them, so you do not have to adjust additional cell IP addresses.

  1. Edit the settings of the NICs to change the IP address to new temporary IPs. 
     vi /etc/sysconfig/network-scripts/ifcfg-eth0

    Where eth0 is the primary IP address for the existing VCD device and it should be updated to a new valid IP.

  2. Restart network services 
     service network restart

Configure a new NFS mount point.

Before you deploy vCloud Director 9.7 we will need to configure an NFS share in your environment. VCD will need root access to this share in order to work and the share must be empty! Here are the steps for configuring an NFS share on a Windows Server. If you are setting up this share on Linux, here is a great from ServerLab on How to setup an NFS Server on CentOS 6.

  1. Install NFS feature.
    1. Launch “Server Manager”
    2. Click on “add roles and features”
    3. Click “next”
    4. Click “Role-based or Feature-based installation” and click “Next”
    5. Select the appropriate server and click “Next”
    6. Scroll down and expand “File and Storage Services” and then expand File and iSCSI services and click on “Server for NFS”
    7. You will probably be presented with additional requirements that need to be installed, simply click ‘Add Features’.
    8. Finish out the wizard to allow the services to install. Reboot the Windows machine when prompted.
  2. Configure new NFS share.
    1. After reboot open Windows Explorer, right click on the C drive and create a new folder.
    2. Name the new folder as you need, for ours we created ‘vcdshare’
    3. Start “Server Manager” and select “File and Storage Services”
    4. Click on “Shares”
    5. Under “TASKS” select “New Share”
    6. A new wizard will display, select “NFS Share - Quick”
    7. Select “Type a custom path” and click the “Browse” button. Locate and click the folder “C:\vcdshare” use the “Select folder” button and click “Next”

      NOTE This folder must be empty!

    8. Enter the share name, ensure the information is correct and click “Next”.
    9. Select “No Server authentication” and ensure “Enable unmapped user access” is selected with “Allow unmapped user access by UID/GID” is selected and click “Next”
    10. On share permissions click “Add”
    11. Enter the eth1 IP address that will be configured on your new appliance. eth1 should be the backend dedicated network for just VCD cell traffic discussion. No other traffic should be on this network, example no vMotion traffic. Ensure the share permissions is set to “Read/Write” and click “Allow root access” and click “Add”. Big thanks to Zach Brown at VMware who pointed out the “Deploying the vCloud Director Appliance” article about the networking design for eth1.
    12. Repeat the last 3 steps to add all cells.
    13. No additional customization is required, simply finish out the wizard to setup the new NFS share.

Deploy new vCloud Director 9.7 primary node.

  1. Download vCloud Director ova file “VMware vCloud Director 9.7.0.4-Virtual Appliance Installation or Migration” from VMware.
  2. In vCenter right click on HA/DRS cluster or hypervisor and select ‘Deploy OVF Template’
  3. In the wizard select ‘local file’ and then ‘choose files’
  4. Locate the downloaded file and click open.
  5. Select ‘Next’
  6. Enter a valid Virtual Machine Name, select the appropriate datacenter to deploy the OVF into, then click ‘Next’.
  7. Select the appropriate hypervisor and click ‘Next’
  8. Review the details and click ‘Next’
  9. Click on the check box for ‘I accept all license agreements’ and then click ‘Next’
  10. Select the appropriate deployment type. For our configuration we selected ‘Primary - large’ (please select the one that is best for you) and then click ‘Next’
  11. Select the appropriate datastore and if the disk is to be thick/thin provisioned and then click ‘Next’
  12. Select the appropriate networks for your configuration and then click ‘Next’
  13. Under Customize Template there is a lot of variables you can enter, enter the correct information in the dialog boxes. Initially ‘All properties have valid values’ will be red but by the end it should turn green. Even though there is a couple that can be skipped I recommend taking the time to fill out all of the information you can. You will notice that you can hardset a highly secure password from the start without having to reset it later. I recommend doing this to ensure the password is set properly from intial deply so enter one that meets the criteria and UNCHECK ‘Expire Root Password Upon First Login’ Example Information: 
    NTP Server to use: time.google.com
Initial Root PW: VMware!1
 Requirements:
 Minimum 8 characters in length
 Minimum 1 uppercase character
 Minimum 1 lowercase character
 Minimum 1 numeric digit
 Minimum 1 special character
 Only visible ASCII characters, including
Expire Root Password Upon First Login: uncheck
Enable SSH root login: check
NFS mount for transfer file location: 10.10.11.10:/vcdshare
NFS must be emtpy!
'vcloud' DB password for the 'vcloud' user: VMware!1
Admin User Name: administrator
Admin Full Name: vCD Admin
Admin user password: VMware!1
Admin email: email@address.here (required)
System name: vcd1
Installation ID: 1
eth0 Network Routes: blank
eth1 Network Routes: blank (enter information if required)
Default Gateway: 10.10.10.1
Domain Name: mv.rackspace.com
Domain Name Servers: 10.10.10.2
eth0 Network IP Address: 10.10.10.11
eth0 Network Netmask: 255.255.255.0
eth1 Network IP Address: 10.10.11.9
eth1 Network Netmask: 255.255.255.0

    NOTE Nearly all information above is required. Domain Search path and Static Routes can be blank

  14. Once all values are entered ‘All properties have valid values’ at the top should be green, then click ‘Next’

    NOTE If you click ‘back’ after this dialog box you might have to re-enter all password information.

  15. Review all of the details, once done click ‘FINISH’
  16. Once the OVA is deployed power it up. I recommend immediately connecting to the device via SSH and looking at the two following files to ensure the appliance was properly configured. If the device was not properly configured it is generally much easier to simply figure out the problem, delete the appliance, and re-deploy it.
    /opt/vmware/var/log/firstboot
    /opt/vmware/var/log/vcd/setupvcd.log
  17. Common errors I experienced are listed here. These errors were always because the NFS mount was not setup properly, did not have root access, or the directory was not empty.

Mounting NFS file share...
System ip is: 10.10.10.10
[ERROR] DB HA type primary should have NFS in clean state. Make sure to delete appliance-nodes dir, cells dir and response file in NFS before retrying this deployment

root@photon-machine [ ~ ]# cat /opt/vmware/var/log/vcd/setupvcd.log
2020-06-05 17:13:22 | Mounting NFS file share...
mount.nfs: access denied by server while mounting 10.10.11.11:/vcdshare
2020-06-05 17:13:22 | [ERROR] Cannot mount NFS transfer location. Ensure that NFS and the mount locations are correct and reachable. Aborting.

VCD Database Availability
No nodes found in cluster, this likely means PostgreSQL is not running on this node. Consult the management UI from another node where PostgreSQL is running. Otherwise, check /opt/vmware/var/log/vcd/vcd_ova_ui_app.log if you think this is an error.

root@photon-machine [ ~ ]# cat /opt/vmware/var/log/firstboot
This script is executed on first boot only.
Setting root password...
Temporarily disabling password strength checks...
New password: Retype new password: passwd: password updated successfully
Re-enabling password strength checks...

Created symlink /etc/systemd/system/multi-user.target.wants/vcd-ova-netconfig.service → /etc/systemd/system/vcd-ova-netconfig.service.
Setting NTP to time.google.com
Invoking vahardening script ...
Invoking generate-certificates script ...
Invoking configure-postgresql script ...
Invoking postgresauth script ...
Generating public/private rsa key pair.
Your identification has been saved in /var/vmware/vpostgres/current/.ssh/id_rsa.
Your public key has been saved in /var/vmware/vpostgres/current/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:###################################
The key's randomart image is:
+---[RSA 4096]----+
| . . |
| E . . . |
| . o o |
| . . . o . |
|* o . S . o .|
|.%.. o.. .. + |
|+oO... . + o..+ .|
|o=.*. . = = .o...|
|. *+. o =o+ . |
+----[SHA256]-----+
Invoking setupvcd script ...
[ERROR] setupvcd script failed to execute. For more details, see: /opt/vmware/var/log/vcd/setupvcd.log

root@photon-machine [ ~ ]# cat /opt/vmware/var/log/vcd/setupvcd.log
2020-06-05 18:24:27 | Mounting NFS file share...
2020-06-05 18:24:27 | System ip0 is: 10.10.10.11
ls: cannot access '/opt/vmware/vcloud-director/data/transfer/appliance-nodes': No such file or directory
ls: cannot access '/opt/vmware/vcloud-director/data/transfer/cells': No such file or directory
2020-06-05 18:24:27 | Invoking appliance-sync.sh now...
2020-06-05 18:24:30 | Configuring vCD from OVF properties as initial cell...
2020-06-05 18:24:30 | Configuring the embedded PostgreSQL database...
2020-06-05 18:24:30 | Creating PostgreSQL user (vcloud) and database (vcloud)...
CREATE ROLE
ALTER ROLE
ALTER ROLE
CREATE DATABASE
2020-06-05 18:24:31 | DB Host value is 10.10.11.11
......................................../
Database configuration complete.

2020-06-05 18:25:07 | Copying local responses.properties to NFS.
2020-06-05 18:25:07 | Invoking firstrunwizard.sh now...
2020-06-05 18:25:10 | [ERROR] Failed to run firstrunwizard.sh. See /opt/vmware/var/log/vcd/firstrunwizard.log for details. Aborting.

Configuring new vPostgres to allow transfer.

  1. On each existing cell and on each newly deployed cell, run the command to quiesce and then stop stop the vCloud Director service. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell -q true
/opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –shutdown
  2. Log into one of the existing cells, this will be the migration source.

    NOTE The device you pick must have access to the eth1 network IP address you configured during the deployment of the appliance.

  3. On the new primary cell you will need to configure Postgres to allow external access from the migration source.
    1. Log into console of the new primary cell as root using the password you specified during the deployment phase above.
    2. Change directories to the database directory 
       cd /opt/vmware/appliance/etc/pg_hba.d/
    3. Copy the existing README file to ‘access.txt’ 
       cp README access.txt
    4. Edit the access.txt file. 
       vi access.txt
    5. Delete the first lines of the file, it should only include the table, like: 
       #TYPE  DATABASE  USER    ADDRESS           METHOD
 host   vcloud    vcloud  172.168.100.5/32  md5
 host   vcloud    vcloud  172.168.20.5/32   md5
    6. Edit the information to include just your IP or IP ranges which need access. The IP information being entered should be the source VCD cell.
    7. The information you enter will be automatically appended to the pg_hba.conf file which acts like an ACL to the primary DB. If needed you can manually restart the Postgres service and wait about a minute for it to update the ACLs. 
       systemctl reload vpostgres.service
    8. After a minute you can confirm they have been added to the pg_hba.conf file automatically with the following command. 
       tail /var/vmware/vpostgres/10/pgdata/pg_hba.conf

      NOTE If you leave the access.txt file in place then Postgres will always use it to apply this set of access controls. If you need to remove the IP then you can edit or remove the file, if you need to open Postgres to another destination IP then it should be in this file. Anything that is in the file is dynamically read and applied.

    9. On the migration source test general connectivity to the new primary cell. This will not test actual DB access but simply if the port is listening between your devices. You should see a response of ‘Connection to 10.10.11.11 5432 port (tcp/postgres) succeeeded!’ response. If you not then troubleshoot your connection issues. 
       nc -z 10.10.11.11 5432

The above method worked for me every time that the appliance was successfully deployed. However, if it did not work for you then you can manually edited the pg_hba.conf file by doing this (not recommended)
1 Locate pg_hba.conf file. My pg_hba.conf file was located in /var/vmware/vpostgres/10/pgdat/
2. Copy the original file for fail-back purposes.
3. Edit the pg_hba.conf file.
4. Add the following line at the bottom:
5. Restart Postgres service
find / -name pg_hba.conf
cp /var/vmware/vpostgres/10/pgdat/pg_hba.conf /var/vmware/vpostgres/10/pgdata/pg_hba.bak
vi /var/vmware/vpostgres/10/pgdata/pg_hba.conf
host vcloud vcloud 10.10.11.8/32 md5
systemctl reload vpostgres.service

If you do not edit the access to the database then you might get the error:

Unable to establish connection to database. Error: Cannot Open Connection
Error executing command: Aborting migration due to IO_ERROR (102)

Import existing data into the new appliance

  1. On the migration source we need to start the migration tool in order to move the DB from existing SQL server into the new embedded postgres server on the appliance. 
    /opt/vmware/vcloud-director/bin/cell-management-tool dbmigrate -dbhost eth1_IP_new_primary -dbport 5432 -dbuser vcloud -dbname vcloud --verbose
  2. When prompted enter your password for the vcloud director database.

/opt/vmware/vcloud-director/bin/cell-management-tool dbmigrate -dbhost 10.10.11.9 -dbport 5432 -dbuser vcloud -dbname vcloud --verbose -dbpassword 'VMware!1'
Configuring the target database...
.......\
Successfully configured the target database.
Target database is using PostgreSQL version 10.6 (VMware Postgres 10.6.0-11022610 release).
Blacklisted licensing_vm_data.
Successfully connected to source DB jdbc:jtds:sqlserver://10.10.11.10:1433/vcloud;socketTimeout=90 and target DB jdbc:postgresql://10.10.11.9:5432/vcloud.
Beginning database migration.
Starting table clearing process.
Successfully cleared table [vapp_property].
Successfully cleared table [gateway_assigned_ip].
Successfully cleared table [address_translation].
Successfully cleared table [guest_personalization_info].
Successfully cleared table [vapp_vm_disk_storage_class].
Successfully cleared table [vapp_eula].
Successfully cleared table [vm_gosc_status].
Successfully cleared table [vapp_product_info].
Successfully cleared table [vapp_vm_sclass_metrics].
Successfully cleared table [gateway_interface].
Successfully cleared table [logical_network_ip_scope].
Successfully cleared table [network_interface].
Successfully cleared table [shared_org_vdc_network].
Successfully cleared table [vdc_lr_encumbrance].
Successfully cleared table [vdc_storage_class_settings].
Successfully cleared table [vapp_logical_resource].
Successfully cleared table [vapp_vm].
Successfully cleared table [real_network_backing].
Successfully cleared table [logical_network].
Successfully cleared table [network_backing].
Successfully cleared table [vdc_logical_resource].
Successfully cleared table [logical_disk].
Successfully cleared table [org_vdc_nsxt_dfw].
Successfully cleared table [org_vdc_network_services].
Successfully cleared table [vdc_network_profile].
Successfully cleared table [vm_container].
Successfully cleared table [vdc_compute_policy_alloc_type].
Successfully cleared table [real_network].
Successfully cleared table [vds_context].
Successfully cleared table [org_prov_vdc_template_bindings].
Successfully cleared table [prov_vdc_logical_resource].
Successfully cleared table [org_prov_vdc].
Successfully cleared table [org_prov_vdc_template_pvdcs].
Successfully cleared table [vdc_compute_policy].
Successfully cleared table [sddc_proxy_certificates].
Successfully cleared table [acl_sddc_proxy].
Successfully cleared table [network_pool].
Successfully cleared table [prov_vdc].
Successfully cleared table [sddc_proxy].
Successfully cleared table [nsxt_manager].
Successfully cleared table [vm_snapshot_disk].
Successfully cleared table [sddcs].
Successfully cleared table [service_apps].
Successfully cleared table [universal_routing_config].
Successfully cleared table [api_acl_rule].
Successfully cleared table [params_soi].
Successfully cleared table [vm_snapshot].
Successfully cleared table [deployed_vm_resource].
Successfully cleared table [drs_rule_vm_inv].
Successfully cleared table [ccr_drs_vm_group_vm_inv].
Successfully cleared table [ccr_drs_vm_group_vm_pending].
Successfully cleared table [ccr_drs_host_group_host_inv].
Successfully cleared table [user_preference].
Successfully cleared table [acl].
Successfully cleared table [custom_entity].
Successfully cleared table [vdc_group_router_egress].
Successfully cleared table [ui_portal_link].
Successfully cleared table [selector_extension_phase].
Successfully cleared table [licensing_ms_data].
Successfully cleared table [ui_plugin_access].
Successfully cleared table [ui_plugin_extension_point].
Successfully cleared table [api_def_supported_versions].
Successfully cleared table [api_file_descriptor].
Successfully cleared table [api_resource].
Successfully cleared table [api_action].
Successfully cleared table [entity_so].
Successfully cleared table [params_so].
Successfully cleared table [params_metainfo_so].
Successfully cleared table [entity_service_item].
Successfully cleared table [acl_service_item].
Successfully cleared table [cell_application_table_access].
Successfully cleared table [shadow_vm].
Successfully cleared table [deployed_vm].
Successfully cleared table [vm_disk_storage_class].
Successfully cleared table [vm_resource].
Successfully cleared table [vm_disk].
Successfully cleared table [vrp_rp].
Successfully cleared table [computevm].
Successfully cleared table [org_prov_vdc_template_access].
Successfully cleared table [ccr_drs_rule_inv].
Successfully cleared table [ccr_drs_vm_host_rule_inv].
Successfully cleared table [ccr_drs_vm_group_inv].
Successfully cleared table [ccr_drs_host_group_inv].
Successfully cleared table [vm_dstore_metrics_inv].
Successfully cleared table [vm_import_activity].
Successfully cleared table [role_proxy_global_role].
Successfully cleared table [org_member].
Successfully cleared table [session_role].
Successfully cleared table [role_right].
Successfully cleared table [global_role_tenant].
Successfully cleared table [rights_bundle_tenant].
Successfully cleared table [vdc_group_cross_vdc_network].
Successfully cleared table [vdc_group_org_vdc].
Successfully cleared table [callout_task_binding].
Successfully cleared table [callout_collaboration].
Successfully cleared table [catalog_item].
Successfully cleared table [ui_portal_branding].
Successfully cleared table [selector_extension].
Successfully cleared table [object_extension_selector].
Successfully cleared table [pvdc_compute_policy_vm_group].
Successfully cleared table [logical_named_vm_group].
Successfully cleared table [cluster_hardware_version].
Successfully cleared table [hardware_version_guest_os].
Successfully cleared table [resource_file].
Successfully cleared table [licensing_metric].
Successfully cleared table [deleted_object_error_detail].
Successfully cleared table [ui_plugin].
Successfully cleared table [api_definition].
Successfully cleared table [api_link].
Successfully cleared table [api_external_service_config].
Successfully cleared table [api_resource_type].
Successfully cleared table [acl_custom_entity_definition].
Successfully cleared table [service_offering].
Successfully cleared table [service].
Successfully cleared table [oidc_scopes].
Successfully cleared table [oauth_key_configuration].
Successfully cleared table [iam_id_provider_settings].
Successfully cleared table [krb_domain_realm_mapping].
Successfully cleared table [pvdc_policy_logical_vm_group].
Successfully cleared table [gateway_assigned_mac].
Successfully cleared table [service_item].
Successfully cleared table [placement_subject_item].
Successfully cleared table [cell_application_instance].
Successfully cleared table [grp].
Successfully cleared table [org_member_source].
Successfully cleared table [job_detail].
Successfully cleared table [last_jobs].
Successfully cleared table [networked_vm_mac].
Successfully cleared table [managed_server].
Successfully cleared table [from_cloud_tunnel].
Successfully cleared table [vm].
Successfully cleared table [disk].
Successfully cleared table [edge_cluster].
Successfully cleared table [tenant_migration_history].
Successfully cleared table [busy_object].
Successfully cleared table [allowed_preference_value].
Successfully cleared table [entity_soi].
Successfully cleared table [deployed_ip_address].
Successfully cleared table [computehub_set_computehub].
Successfully cleared table [vrp].
Successfully cleared table [org_prov_vdc_specification_nw].
Successfully cleared table [org_prov_vdc_template_sprofile].
Successfully cleared table [org_prov_vdc_template].
Successfully cleared table [metadata_string_value].
Successfully cleared table [metadata_boolean_value].
Successfully cleared table [metadata_long_value].
Successfully cleared table [metadata_text_value].
Successfully cleared table [metadata_date_value].
Successfully cleared table [remote_orgs].
Successfully cleared table [site_status].
Successfully cleared table [org_association].
Successfully cleared table [universal_transport_zone].
Successfully cleared table [rights_implications].
Successfully cleared table [ip_range].
Successfully cleared table [allocated_ip_address].
Successfully cleared table [datastore].
Successfully cleared table [vc_computehub].
Successfully cleared table [folder_inv].
Successfully cleared table [vc_stat_runtime].
Successfully cleared table [resource_pool_inv].
Successfully cleared table [custom_field_manager_inv].
Successfully cleared table [network_inv].
Successfully cleared table [property_map].
Successfully cleared table [managed_server_inv].
Successfully cleared table [datacenter_inv].
Successfully cleared table [compute_resource_inv].
Successfully cleared table [storage_pod_inv].
Successfully cleared table [cluster_compute_resource_inv].
Successfully cleared table [vm_inv].
Successfully cleared table [dv_switch].
Successfully cleared table [dv_switch_inv].
Successfully cleared table [vshield_manager].
Successfully cleared table [dv_portgroup_inv].
Successfully cleared table [vcenter_runtime].
Successfully cleared table [datastore_inv].
Successfully cleared table [role].
Successfully cleared table [vdc_group_logical_resource].
Successfully cleared table [smtp_server].
Successfully cleared table [callout_endpoint].
Successfully cleared table [org_email].
Successfully cleared table [catalog].
Successfully cleared table [branding].
Successfully cleared table [org_association_keys].
Successfully cleared table [task_activity_queue].
Successfully cleared table [datacenter_network_inv].
Successfully cleared table [vc_activity_queue].
Successfully cleared table [storage_fabric_handle].
Successfully cleared table [federated_user].
Successfully cleared table [managed_server_network_inv].
Successfully cleared table [org_delegation_user].
Successfully cleared table [dv_switch_resource].
Successfully cleared table [vc_task_mapping].
Successfully cleared table [ldisk_storage_class_join].
Successfully cleared table [network_resources_config].
Successfully cleared table [lock_handle].
Successfully cleared table [storage_profile_inv].
Successfully cleared table [group_user].
Successfully cleared table [metric_configuration].
Successfully cleared table [object_condition].
Successfully cleared table [identity_provider].
Successfully cleared table [activity_partition_23].
Successfully cleared table [network_fabric_handle].
Successfully cleared table [activity_partition_12].
Successfully cleared table [managed_media].
Successfully cleared table [vro_server].
Successfully cleared table [activity_partition_7].
Successfully cleared table [audit_event].
Successfully cleared table [event].
Successfully cleared table [ui_portal_resource].
Successfully cleared table [shared_resource_assignment].
Successfully cleared table [activity_partition_26].
Successfully cleared table [activity_partition_6].
Successfully cleared table [activity_partition_27].
Successfully cleared table [activity_partition_18].
Successfully cleared table [activity_partition_9].
Successfully cleared table [activity_partition_17].
Successfully cleared table [activity_partition_1].
Successfully cleared table [activity_stats_queue].
Successfully cleared table [opaque_network_inv].
Successfully cleared table [object_extension].
Successfully cleared table [activity_partition_20].
Successfully cleared table [transfer_server].
Successfully cleared table [activity_partition_2].
Successfully cleared table [activity_partition_29].
Successfully cleared table [failed_cells].
Successfully cleared table [acl_custom_entity].
Successfully cleared table [activity_partition_11].
Successfully cleared table [activity_entity_record].
Successfully cleared table [activity].
Successfully cleared table [activity_partition_3].
Successfully cleared table [activity_partition_19].
Successfully cleared table [activity_partition_4].
Successfully cleared table [activity_partition_14].
Successfully cleared table [named_vm_group].
Successfully cleared table [cluster_compute_resource_ext].
Successfully cleared table [hardware_version].
Successfully cleared table [lock_intent].
Successfully cleared table [scheduled_activity_jobs].
Successfully cleared table [activity_partition_16].
Successfully cleared table [amqp].
Successfully cleared table [folder].
Successfully cleared table [transfer_session].
Successfully cleared table [federated_group].
Successfully cleared table [activity_partition_30].
Successfully cleared table [tunneling_application].
Successfully cleared table [activity_partition_22].
Successfully cleared table [reservation].
Successfully cleared table [activity_partition_13].
Successfully cleared table [activity_partition_24].
Successfully cleared table [licensing_report].
Successfully cleared table [ui_plugin_installation].
Successfully cleared table [activity_partition_5].
Successfully cleared table [deleted_object].
Successfully cleared table [sprofile_compliance].
Successfully cleared table [valc_storage_class_join].
Successfully cleared table [monitored_entity].
Successfully cleared table [config].
Successfully cleared table [ui_plugin_resource].
Successfully cleared table [activity_partition_28].
Successfully cleared table [api_external_service].
Successfully cleared table [custom_entity_definition].
Successfully cleared table [extension].
Successfully cleared table [disk_adapter_type].
Successfully cleared table [task_inv].
Successfully cleared table [gateway_logical_resource].
Successfully cleared table [activity_partition_8].
Successfully cleared table [activity_partition_10].
Successfully cleared table [vapp_vm_logical_disk].
Successfully cleared table [activity_vsm_listener_queue].
Successfully cleared table [oauth_id_provider_settings].
Successfully cleared table [activity_partition_31].
Successfully cleared table [krb_realm_config].
Successfully cleared table [activity_partition_15].
Successfully cleared table [logical_vm_group].
Successfully cleared table [activity_pc_queue].
Successfully cleared table [gateway].
Successfully cleared table [saml_id_provider_settings].
Successfully cleared table [service_library].
Successfully cleared table [datastore_profile_inv].
Successfully cleared table [placement_subject].
Successfully cleared table [cells].
Successfully cleared table [member_source].
Successfully cleared table [managed_server_datastore_inv].
Successfully cleared table [activity_partition_21].
Successfully cleared table [ldap_provider].
Successfully cleared table [truststore].
Successfully cleared table [ldisk_fo_join].
Successfully cleared table [activity_partition_25].
Successfully cleared table [ext_localization_resources].
Successfully cleared table [extensibility_selector].
Successfully cleared table [network_pool_merge_data].
Successfully cleared table [compute_fabric_handle].
Successfully cleared table [federation_settings].
Successfully cleared table [jobs].
Successfully cleared table [resource_item].
Successfully cleared table [from_cloud_tunnel_listener].
Successfully cleared table [storage_class].
Successfully cleared table [library_storage_profiles].
Successfully cleared table [task].
Successfully cleared table [cloud_proxy_ticket].
Successfully cleared table [resource_assignment].
Successfully cleared table [preference].
Successfully cleared table [pvdc_compute_policy].
Successfully cleared table [ui_portal_theme].
Successfully cleared table [service_offering_instance].
Successfully cleared table [os].
Successfully cleared table [user_session].
Successfully cleared table [reserved_ip_address].
Successfully cleared table [extensibility_phase].
Successfully cleared table [computehub_set].
Successfully cleared table [guest_osfamily].
Successfully cleared table [ext_localization_bundles].
Successfully cleared table [object_config].
Successfully cleared table [fifo_activity_queue].
Successfully cleared table [usr].
Successfully cleared table [ui_extension_point].
Successfully cleared table [vcenter_supported_builds].
Successfully cleared table [api_idempotent_store].
Successfully cleared table [org_prov_vdc_specification].
Successfully cleared table [to_cloud_tunnel].
Successfully cleared table [metadata].
Successfully cleared table [sites].
Successfully cleared table [rights].
Successfully cleared table [ip_scope].
Successfully cleared table [networked_vm].
Successfully cleared table [activity_partition_0].
Successfully cleared table [virtual_center].
Successfully cleared table [organization].
Successfully cleared table [saml_message_storage].
Finished clearing tables for migration. Beginning database migration.
Migrating table [saml_message_storage].
Successfully migrated table [saml_message_storage] : (0)
Migrating table [organization].
Successfully migrated table [organization] : (5)
Migrating table [virtual_center].
Successfully migrated table [virtual_center] : (1)
Migrating table [activity_partition_0].
Successfully migrated table [activity_partition_0] : (0)
Migrating table [networked_vm].
Successfully migrated table [networked_vm] : (1,421)
Migrating table [ip_scope].
Successfully migrated table [ip_scope] : (178)
Migrating table [rights].
Successfully migrated table [rights] : (319)
Migrating table [sites].
Successfully migrated table [sites] : (1)
Migrating table [metadata].
Successfully migrated table [metadata] : (6,099)
Migrating table [to_cloud_tunnel].
Successfully migrated table [to_cloud_tunnel] : (0)
Migrating table [org_prov_vdc_specification].
Successfully migrated table [org_prov_vdc_specification] : (0)
Migrating table [api_idempotent_store].
Successfully migrated table [api_idempotent_store] : (0)
Migrating table [vcenter_supported_builds].
Successfully migrated table [vcenter_supported_builds] : (6)
Migrating table [ui_extension_point].
Successfully migrated table [ui_extension_point] : (2)
Migrating table [usr].
Successfully migrated table [usr] : (126)
Migrating table [fifo_activity_queue].
Successfully migrated table [fifo_activity_queue] : (0)
Migrating table [object_config].
Successfully migrated table [object_config] : (79)
Migrating table [ext_localization_bundles].
Successfully migrated table [ext_localization_bundles] : (0)
Migrating table [guest_osfamily].
Successfully migrated table [guest_osfamily] : (3)
Migrating table [computehub_set].
Successfully migrated table [computehub_set] : (9)
Migrating table [extensibility_phase].
Successfully migrated table [extensibility_phase] : (10)
Migrating table [reserved_ip_address].
Successfully migrated table [reserved_ip_address] : (0)
Migrating table [user_session].
Successfully migrated table [user_session] : (0)
Migrating table [os].
Successfully migrated table [os] : (23)
Migrating table [service_offering_instance].
Successfully migrated table [service_offering_instance] : (0)
Migrating table [ui_portal_theme].
Successfully migrated table [ui_portal_theme] : (2)
Migrating table [pvdc_compute_policy].
Successfully migrated table [pvdc_compute_policy] : (0)
Migrating table [preference].
Successfully migrated table [preference] : (4)
Migrating table [resource_assignment].
Successfully migrated table [resource_assignment] : (308)
Migrating table [cloud_proxy_ticket].
Successfully migrated table [cloud_proxy_ticket] : (0)
Migrating table [task].
Successfully migrated table [task] : (0)
Migrating table [library_storage_profiles].
Successfully migrated table [library_storage_profiles] : (5)
Migrating table [storage_class].
Successfully migrated table [storage_class] : (10)
Migrating table [from_cloud_tunnel_listener].
Successfully migrated table [from_cloud_tunnel_listener] : (0)
Migrating table [resource_item].
Successfully migrated table [resource_item] : (69,644)
Migrating table [jobs].
Successfully migrated table [jobs] : (19,436)
Migrating table [federation_settings].
Successfully migrated table [federation_settings] : (6)
Migrating table [compute_fabric_handle].
Successfully migrated table [compute_fabric_handle] : (1,599)
Migrating table [network_pool_merge_data].
Successfully migrated table [network_pool_merge_data] : (0)
Migrating table [extensibility_selector].
Successfully migrated table [extensibility_selector] : (4)
Migrating table [ext_localization_resources].
Successfully migrated table [ext_localization_resources] : (0)
Migrating table [activity_partition_25].
Successfully migrated table [activity_partition_25] : (0)
Migrating table [ldisk_fo_join].
Successfully migrated table [ldisk_fo_join] : (3)
Migrating table [truststore].
Successfully migrated table [truststore] : (5)
Migrating table [ldap_provider].
Successfully migrated table [ldap_provider] : (1)
Migrating table [activity_partition_21].
Successfully migrated table [activity_partition_21] : (0)
Migrating table [managed_server_datastore_inv].
Successfully migrated table [managed_server_datastore_inv] : (1,467)
Migrating table [member_source].
Successfully migrated table [member_source] : (7)
Migrating table [cells].
Successfully migrated table [cells] : (1)
Migrating table [placement_subject].
Successfully migrated table [placement_subject] : (0)
Migrating table [datastore_profile_inv].
Successfully migrated table [datastore_profile_inv] : (245)
Migrating table [service_library].
Successfully migrated table [service_library] : (0)
Migrating table [saml_id_provider_settings].
Successfully migrated table [saml_id_provider_settings] : (1)
Migrating table [gateway].
Successfully migrated table [gateway] : (18)
Migrating table [activity_pc_queue].
Successfully migrated table [activity_pc_queue] : (2)
Migrating table [logical_vm_group].
Successfully migrated table [logical_vm_group] : (0)
Migrating table [activity_partition_15].
Successfully migrated table [activity_partition_15] : (0)
Migrating table [krb_realm_config].
Successfully migrated table [krb_realm_config] : (0)
Migrating table [activity_partition_31].
Successfully migrated table [activity_partition_31] : (0)
Migrating table [oauth_id_provider_settings].
Successfully migrated table [oauth_id_provider_settings] : (0)
Migrating table [activity_vsm_listener_queue].
Successfully migrated table [activity_vsm_listener_queue] : (0)
Migrating table [vapp_vm_logical_disk].
Successfully migrated table [vapp_vm_logical_disk] : (1)
Migrating table [activity_partition_10].
Successfully migrated table [activity_partition_10] : (0)
Migrating table [activity_partition_8].
Successfully migrated table [activity_partition_8] : (0)
Migrating table [gateway_logical_resource].
Successfully migrated table [gateway_logical_resource] : (0)
Migrating table [task_inv].
Successfully migrated table [task_inv] : (5,599)
Migrating table [disk_adapter_type].
Successfully migrated table [disk_adapter_type] : (6)
Migrating table [extension].
Successfully migrated table [extension] : (0)
Migrating table [custom_entity_definition].
Successfully migrated table [custom_entity_definition] : (2)
Migrating table [api_external_service].
Successfully migrated table [api_external_service] : (0)
Migrating table [activity_partition_28].
Successfully migrated table [activity_partition_28] : (0)
Migrating table [ui_plugin_resource].
Successfully migrated table [ui_plugin_resource] : (3)
Migrating table [config].
Successfully migrated table [config] : (327)
Migrating table [monitored_entity].
Successfully migrated table [monitored_entity] : (0)
Migrating table [valc_storage_class_join].
Successfully migrated table [valc_storage_class_join] : (52)
Migrating table [sprofile_compliance].
Successfully migrated table [sprofile_compliance] : (526)
Migrating table [deleted_object].
Successfully migrated table [deleted_object] : (4)
Migrating table [activity_partition_5].
Successfully migrated table [activity_partition_5] : (0)
Migrating table [ui_plugin_installation].
Successfully migrated table [ui_plugin_installation] : (3)
Migrating table [licensing_report].
Successfully migrated table [licensing_report] : (50)
Migrating table [activity_partition_24].
Successfully migrated table [activity_partition_24] : (0)
Migrating table [activity_partition_13].
Successfully migrated table [activity_partition_13] : (0)
Migrating table [reservation].
Successfully migrated table [reservation] : (0)
Migrating table [activity_partition_22].
Successfully migrated table [activity_partition_22] : (0)
Migrating table [tunneling_application].
Successfully migrated table [tunneling_application] : (0)
Migrating table [activity_partition_30].
Successfully migrated table [activity_partition_30] : (0)
Migrating table [federated_group].
Successfully migrated table [federated_group] : (0)
Migrating table [transfer_session].
Successfully migrated table [transfer_session] : (0)
Migrating table [folder].
Successfully migrated table [folder] : (0)
Migrating table [amqp].
Successfully migrated table [amqp] : (1)
Migrating table [activity_partition_16].
Successfully migrated table [activity_partition_16] : (0)
Migrating table [scheduled_activity_jobs].
Successfully migrated table [scheduled_activity_jobs] : (0)
Migrating table [lock_intent].
Successfully migrated table [lock_intent] : (0)
Migrating table [hardware_version].
Successfully migrated table [hardware_version] : (11)
Migrating table [cluster_compute_resource_ext].
Successfully migrated table [cluster_compute_resource_ext] : (14)
Migrating table [named_vm_group].
Successfully migrated table [named_vm_group] : (19)
Migrating table [activity_partition_14].
Successfully migrated table [activity_partition_14] : (0)
Migrating table [activity_partition_4].
Successfully migrated table [activity_partition_4] : (0)
Migrating table [activity_partition_19].
Successfully migrated table [activity_partition_19] : (0)
Migrating table [activity_partition_3].
Successfully migrated table [activity_partition_3] : (0)
Migrating table [activity].
Successfully migrated table [activity] : (2,687)
Migrating table [activity_entity_record].
Successfully migrated table [activity_entity_record] : (0)
Migrating table [activity_partition_11].
Successfully migrated table [activity_partition_11] : (0)
Migrating table [acl_custom_entity].
Successfully migrated table [acl_custom_entity] : (0)
Migrating table [failed_cells].
Successfully migrated table [failed_cells] : (0)
Migrating table [activity_partition_29].
Successfully migrated table [activity_partition_29] : (0)
Migrating table [activity_partition_2].
Successfully migrated table [activity_partition_2] : (0)
Migrating table [transfer_server].
Successfully migrated table [transfer_server] : (0)
Migrating table [activity_partition_20].
Successfully migrated table [activity_partition_20] : (0)
Migrating table [object_extension].
Successfully migrated table [object_extension] : (0)
Migrating table [opaque_network_inv].
Successfully migrated table [opaque_network_inv] : (0)
Migrating table [activity_stats_queue].
Successfully migrated table [activity_stats_queue] : (0)
Migrating table [activity_partition_1].
Successfully migrated table [activity_partition_1] : (0)
Migrating table [activity_partition_17].
Successfully migrated table [activity_partition_17] : (0)
Migrating table [activity_partition_9].
Successfully migrated table [activity_partition_9] : (0)
Migrating table [activity_partition_18].
Successfully migrated table [activity_partition_18] : (0)
Migrating table [activity_partition_27].
Successfully migrated table [activity_partition_27] : (0)
Migrating table [activity_partition_6].
Successfully migrated table [activity_partition_6] : (0)
Migrating table [activity_partition_26].
Successfully migrated table [activity_partition_26] : (0)
Migrating table [shared_resource_assignment].
Successfully migrated table [shared_resource_assignment] : (0)
Migrating table [ui_portal_resource].
Successfully migrated table [ui_portal_resource] : (0)
Migrating table [event].
Successfully migrated table [event] : (0)
Migrating table [audit_event].
[audit_event] - 42% migrated.
[audit_event] - 85% migrated.
Successfully migrated table [audit_event] : (233,893)
Migrating table [activity_partition_7].
Successfully migrated table [activity_partition_7] : (0)
Migrating table [vro_server].
Successfully migrated table [vro_server] : (0)
Migrating table [managed_media].
Successfully migrated table [managed_media] : (4)
Migrating table [activity_partition_12].
Successfully migrated table [activity_partition_12] : (0)
Migrating table [network_fabric_handle].
Successfully migrated table [network_fabric_handle] : (4,242)
Migrating table [activity_partition_23].
Successfully migrated table [activity_partition_23] : (0)
Migrating table [identity_provider].
Successfully migrated table [identity_provider] : (6)
Migrating table [object_condition].
Successfully migrated table [object_condition] : (860)
Migrating table [metric_configuration].
Successfully migrated table [metric_configuration] : (8)
Migrating table [group_user].
Successfully migrated table [group_user] : (219)
Migrating table [storage_profile_inv].
Successfully migrated table [storage_profile_inv] : (13)
Migrating table [lock_handle].
Successfully migrated table [lock_handle] : (0)
Migrating table [network_resources_config].
Successfully migrated table [network_resources_config] : (1)
Migrating table [ldisk_storage_class_join].
Successfully migrated table [ldisk_storage_class_join] : (3)
Migrating table [vc_task_mapping].
Successfully migrated table [vc_task_mapping] : (4,670)
Migrating table [dv_switch_resource].
Successfully migrated table [dv_switch_resource] : (0)
Migrating table [org_delegation_user].
Successfully migrated table [org_delegation_user] : (0)
Migrating table [managed_server_network_inv].
Successfully migrated table [managed_server_network_inv] : (10,776)
Migrating table [federated_user].
Successfully migrated table [federated_user] : (0)
Migrating table [storage_fabric_handle].
Successfully migrated table [storage_fabric_handle] : (189)
Migrating table [vc_activity_queue].
Successfully migrated table [vc_activity_queue] : (0)
Migrating table [datacenter_network_inv].
Successfully migrated table [datacenter_network_inv] : (238)
Migrating table [task_activity_queue].
Successfully migrated table [task_activity_queue] : (1,559)
Migrating table [org_association_keys].
Successfully migrated table [org_association_keys] : (5)
Migrating table [branding].
Successfully migrated table [branding] : (1)
Migrating table [catalog].
Successfully migrated table [catalog] : (8)
Migrating table [org_email].
Successfully migrated table [org_email] : (5)
Migrating table [callout_endpoint].
Successfully migrated table [callout_endpoint] : (1)
Migrating table [smtp_server].
Successfully migrated table [smtp_server] : (5)
Migrating table [vdc_group_logical_resource].
Successfully migrated table [vdc_group_logical_resource] : (0)
Migrating table [role].
Successfully migrated table [role] : (54)
Migrating table [datastore_inv].
Successfully migrated table [datastore_inv] : (246)
Migrating table [vcenter_runtime].
Successfully migrated table [vcenter_runtime] : (2)
Migrating table [dv_portgroup_inv].
Successfully migrated table [dv_portgroup_inv] : (190)
Migrating table [vshield_manager].
Successfully migrated table [vshield_manager] : (1)
Migrating table [dv_switch_inv].
Successfully migrated table [dv_switch_inv] : (14)
Migrating table [dv_switch].
Successfully migrated table [dv_switch] : (0)
Migrating table [vm_inv].
Successfully migrated table [vm_inv] : (1,467)
Migrating table [cluster_compute_resource_inv].
Successfully migrated table [cluster_compute_resource_inv] : (14)
Migrating table [storage_pod_inv].
Successfully migrated table [storage_pod_inv] : (5)
Migrating table [compute_resource_inv].
Successfully migrated table [compute_resource_inv] : (1)
Migrating table [datacenter_inv].
Successfully migrated table [datacenter_inv] : (7)
Migrating table [managed_server_inv].
Successfully migrated table [managed_server_inv] : (203)
Migrating table [property_map].
Successfully migrated table [property_map] : (3,902)
Migrating table [network_inv].
Successfully migrated table [network_inv] : (36)
Migrating table [custom_field_manager_inv].
Successfully migrated table [custom_field_manager_inv] : (1)
Migrating table [resource_pool_inv].
Successfully migrated table [resource_pool_inv] : (177)
Migrating table [vc_stat_runtime].
Successfully migrated table [vc_stat_runtime] : (1)
Migrating table [folder_inv].
Successfully migrated table [folder_inv] : (1,541)
Migrating table [vc_computehub].
Successfully migrated table [vc_computehub] : (9)
Migrating table [datastore].
Successfully migrated table [datastore] : (228)
Migrating table [allocated_ip_address].
Successfully migrated table [allocated_ip_address] : (1,412)
Migrating table [ip_range].
Successfully migrated table [ip_range] : (180)
Migrating table [rights_implications].
Successfully migrated table [rights_implications] : (0)
Migrating table [universal_transport_zone].
Successfully migrated table [universal_transport_zone] : (0)
Migrating table [org_association].
Successfully migrated table [org_association] : (0)
Migrating table [site_status].
Successfully migrated table [site_status] : (0)
Migrating table [remote_orgs].
Successfully migrated table [remote_orgs] : (0)
Migrating table [metadata_date_value].
Successfully migrated table [metadata_date_value] : (0)
Migrating table [metadata_text_value].
Successfully migrated table [metadata_text_value] : (0)
Migrating table [metadata_long_value].
Successfully migrated table [metadata_long_value] : (1)
Migrating table [metadata_boolean_value].
Successfully migrated table [metadata_boolean_value] : (3)
Migrating table [metadata_string_value].
Successfully migrated table [metadata_string_value] : (6,095)
Migrating table [org_prov_vdc_template].
Successfully migrated table [org_prov_vdc_template] : (0)
Migrating table [org_prov_vdc_template_sprofile].
Successfully migrated table [org_prov_vdc_template_sprofile] : (0)
Migrating table [org_prov_vdc_specification_nw].
Successfully migrated table [org_prov_vdc_specification_nw] : (0)
Migrating table [vrp].
Successfully migrated table [vrp] : (156)
Migrating table [computehub_set_computehub].
Successfully migrated table [computehub_set_computehub] : (9)
Migrating table [deployed_ip_address].
Successfully migrated table [deployed_ip_address] : (0)
Migrating table [entity_soi].
Successfully migrated table [entity_soi] : (0)
Migrating table [allowed_preference_value].
Successfully migrated table [allowed_preference_value] : (0)
Migrating table [busy_object].
Successfully migrated table [busy_object] : (0)
Migrating table [tenant_migration_history].
Successfully migrated table [tenant_migration_history] : (0)
Migrating table [edge_cluster].
Successfully migrated table [edge_cluster] : (0)
Migrating table [disk].
Successfully migrated table [disk] : (0)
Migrating table [vm].
Successfully migrated table [vm] : (1,467)
Migrating table [from_cloud_tunnel].
Successfully migrated table [from_cloud_tunnel] : (0)
Migrating table [managed_server].
Successfully migrated table [managed_server] : (156)
Migrating table [networked_vm_mac].
Successfully migrated table [networked_vm_mac] : (1,439)
Migrating table [last_jobs].
Successfully migrated table [last_jobs] : (384)
Migrating table [job_detail].
Successfully migrated table [job_detail] : (17,390)
Migrating table [org_member_source].
Successfully migrated table [org_member_source] : (11)
Migrating table [grp].
Successfully migrated table [grp] : (14)
Migrating table [cell_application_instance].
Successfully migrated table [cell_application_instance] : (5)
Migrating table [placement_subject_item].
Successfully migrated table [placement_subject_item] : (0)
Migrating table [service_item].
Successfully migrated table [service_item] : (0)
Migrating table [gateway_assigned_mac].
Successfully migrated table [gateway_assigned_mac] : (0)
Migrating table [pvdc_policy_logical_vm_group].
Successfully migrated table [pvdc_policy_logical_vm_group] : (0)
Migrating table [krb_domain_realm_mapping].
Successfully migrated table [krb_domain_realm_mapping] : (0)
Migrating table [iam_id_provider_settings].
Successfully migrated table [iam_id_provider_settings] : (0)
Migrating table [oauth_key_configuration].
Successfully migrated table [oauth_key_configuration] : (0)
Migrating table [oidc_scopes].
Successfully migrated table [oidc_scopes] : (0)
Migrating table [service].
Successfully migrated table [service] : (0)
Migrating table [service_offering].
Successfully migrated table [service_offering] : (0)
Migrating table [acl_custom_entity_definition].
Successfully migrated table [acl_custom_entity_definition] : (0)
Migrating table [api_resource_type].
Successfully migrated table [api_resource_type] : (0)
Migrating table [api_external_service_config].
Successfully migrated table [api_external_service_config] : (0)
Migrating table [api_link].
Successfully migrated table [api_link] : (0)
Migrating table [api_definition].
Successfully migrated table [api_definition] : (0)
Migrating table [ui_plugin].
Successfully migrated table [ui_plugin] : (3)
Migrating table [deleted_object_error_detail].
Successfully migrated table [deleted_object_error_detail] : (4)
Migrating table [licensing_metric].
Successfully migrated table [licensing_metric] : (35,355)
Migrating table [resource_file].
Successfully migrated table [resource_file] : (0)
Migrating table [hardware_version_guest_os].
Successfully migrated table [hardware_version_guest_os] : (1,059)
Migrating table [cluster_hardware_version].
Successfully migrated table [cluster_hardware_version] : (19)
Migrating table [logical_named_vm_group].
Successfully migrated table [logical_named_vm_group] : (0)
Migrating table [pvdc_compute_policy_vm_group].
Successfully migrated table [pvdc_compute_policy_vm_group] : (0)
Migrating table [object_extension_selector].
Successfully migrated table [object_extension_selector] : (0)
Migrating table [selector_extension].
Successfully migrated table [selector_extension] : (0)
Migrating table [ui_portal_branding].
Successfully migrated table [ui_portal_branding] : (1)
Migrating table [catalog_item].
Successfully migrated table [catalog_item] : (55)
Migrating table [callout_collaboration].
Successfully migrated table [callout_collaboration] : (0)
Migrating table [callout_task_binding].
Successfully migrated table [callout_task_binding] : (0)
Migrating table [vdc_group_org_vdc].
Successfully migrated table [vdc_group_org_vdc] : (0)
Migrating table [vdc_group_cross_vdc_network].
Successfully migrated table [vdc_group_cross_vdc_network] : (0)
Migrating table [rights_bundle_tenant].
Successfully migrated table [rights_bundle_tenant] : (5)
Migrating table [global_role_tenant].
Successfully migrated table [global_role_tenant] : (32)
Migrating table [role_right].
Successfully migrated table [role_right] : (1,865)
Migrating table [session_role].
Successfully migrated table [session_role] : (0)
Migrating table [org_member].
Successfully migrated table [org_member] : (192)
Migrating table [role_proxy_global_role].
Successfully migrated table [role_proxy_global_role] : (32)
Migrating table [vm_import_activity].
Successfully migrated table [vm_import_activity] : (2)
Migrating table [vm_dstore_metrics_inv].
Successfully migrated table [vm_dstore_metrics_inv] : (1,474)
Migrating table [ccr_drs_host_group_inv].
Successfully migrated table [ccr_drs_host_group_inv] : (5)
Migrating table [ccr_drs_vm_group_inv].
Successfully migrated table [ccr_drs_vm_group_inv] : (12)
Migrating table [ccr_drs_vm_host_rule_inv].
Successfully migrated table [ccr_drs_vm_host_rule_inv] : (0)
Migrating table [ccr_drs_rule_inv].
Successfully migrated table [ccr_drs_rule_inv] : (0)
Migrating table [org_prov_vdc_template_access].
Successfully migrated table [org_prov_vdc_template_access] : (0)
Migrating table [computevm].
Successfully migrated table [computevm] : (1,434)
Migrating table [vrp_rp].
Successfully migrated table [vrp_rp] : (156)
Migrating table [vm_disk].
Successfully migrated table [vm_disk] : (0)
Migrating table [vm_resource].
Successfully migrated table [vm_resource] : (0)
Migrating table [vm_disk_storage_class].
Successfully migrated table [vm_disk_storage_class] : (20)
Migrating table [deployed_vm].
Successfully migrated table [deployed_vm] : (1,298)
Migrating table [shadow_vm].
Successfully migrated table [shadow_vm] : (12)
Migrating table [cell_application_table_access].
Successfully migrated table [cell_application_table_access] : (1,200)
Migrating table [acl_service_item].
Successfully migrated table [acl_service_item] : (0)
Migrating table [entity_service_item].
Successfully migrated table [entity_service_item] : (0)
Migrating table [params_metainfo_so].
Successfully migrated table [params_metainfo_so] : (0)
Migrating table [params_so].
Successfully migrated table [params_so] : (0)
Migrating table [entity_so].
Successfully migrated table [entity_so] : (0)
Migrating table [api_action].
Successfully migrated table [api_action] : (0)
Migrating table [api_resource].
Successfully migrated table [api_resource] : (0)
Migrating table [api_file_descriptor].
Successfully migrated table [api_file_descriptor] : (0)
Migrating table [api_def_supported_versions].
Successfully migrated table [api_def_supported_versions] : (0)
Migrating table [ui_plugin_extension_point].
Successfully migrated table [ui_plugin_extension_point] : (0)
Migrating table [ui_plugin_access].
Successfully migrated table [ui_plugin_access] : (0)
Migrating table [licensing_ms_data].
[licensing_ms_data] - 2% migrated.
[licensing_ms_data] - 5% migrated.
[licensing_ms_data] - 8% migrated.
[licensing_ms_data] - 11% migrated.
[licensing_ms_data] - 14% migrated.
[licensing_ms_data] - 17% migrated.
[licensing_ms_data] - 20% migrated.
[licensing_ms_data] - 22% migrated.
[licensing_ms_data] - 25% migrated.
[licensing_ms_data] - 28% migrated.
[licensing_ms_data] - 31% migrated.
[licensing_ms_data] - 34% migrated.
[licensing_ms_data] - 37% migrated.
[licensing_ms_data] - 40% migrated.
[licensing_ms_data] - 43% migrated.
[licensing_ms_data] - 45% migrated.
[licensing_ms_data] - 48% migrated.
[licensing_ms_data] - 51% migrated.
[licensing_ms_data] - 54% migrated.
[licensing_ms_data] - 57% migrated.
[licensing_ms_data] - 60% migrated.
[licensing_ms_data] - 63% migrated.
[licensing_ms_data] - 66% migrated.
[licensing_ms_data] - 68% migrated.
[licensing_ms_data] - 71% migrated.
[licensing_ms_data] - 74% migrated.
[licensing_ms_data] - 77% migrated.
[licensing_ms_data] - 80% migrated.
[licensing_ms_data] - 83% migrated.
[licensing_ms_data] - 86% migrated.
[licensing_ms_data] - 89% migrated.
[licensing_ms_data] - 91% migrated.
[licensing_ms_data] - 94% migrated.
[licensing_ms_data] - 97% migrated.
Successfully migrated table [licensing_ms_data] : (3,478,267)
Migrating table [selector_extension_phase].
Successfully migrated table [selector_extension_phase] : (0)
Migrating table [ui_portal_link].
Successfully migrated table [ui_portal_link] : (0)
Migrating table [vdc_group_router_egress].
Successfully migrated table [vdc_group_router_egress] : (0)
Migrating table [custom_entity].
Successfully migrated table [custom_entity] : (0)
Migrating table [acl].
Successfully migrated table [acl] : (166)
Migrating table [user_preference].
Successfully migrated table [user_preference] : (10)
Migrating table [ccr_drs_host_group_host_inv].
Successfully migrated table [ccr_drs_host_group_host_inv] : (16)
Migrating table [ccr_drs_vm_group_vm_pending].
Successfully migrated table [ccr_drs_vm_group_vm_pending] : (0)
Migrating table [ccr_drs_vm_group_vm_inv].
Successfully migrated table [ccr_drs_vm_group_vm_inv] : (0)
Migrating table [drs_rule_vm_inv].
Successfully migrated table [drs_rule_vm_inv] : (0)
Migrating table [deployed_vm_resource].
Successfully migrated table [deployed_vm_resource] : (0)
Migrating table [vm_snapshot].
Successfully migrated table [vm_snapshot] : (19)
Migrating table [params_soi].
Successfully migrated table [params_soi] : (0)
Migrating table [api_acl_rule].
Successfully migrated table [api_acl_rule] : (0)
Migrating table [universal_routing_config].
Successfully migrated table [universal_routing_config] : (0)
Migrating table [service_apps].
Successfully migrated table [service_apps] : (0)
Migrating table [sddcs].
Successfully migrated table [sddcs] : (0)
Migrating table [vm_snapshot_disk].
Successfully migrated table [vm_snapshot_disk] : (11)
Migrating table [nsxt_manager].
Successfully migrated table [nsxt_manager] : (0)
Migrating table [sddc_proxy].
Successfully migrated table [sddc_proxy] : (0)
Migrating table [prov_vdc].
Successfully migrated table [prov_vdc] : (9)
Migrating table [network_pool].
Successfully migrated table [network_pool] : (9)
Migrating table [acl_sddc_proxy].
Successfully migrated table [acl_sddc_proxy] : (0)
Migrating table [sddc_proxy_certificates].
Successfully migrated table [sddc_proxy_certificates] : (0)
Migrating table [vdc_compute_policy].
Successfully migrated table [vdc_compute_policy] : (243)
Migrating table [org_prov_vdc_template_pvdcs].
Successfully migrated table [org_prov_vdc_template_pvdcs] : (0)
Migrating table [org_prov_vdc].
Successfully migrated table [org_prov_vdc] : (151)
Migrating table [prov_vdc_logical_resource].
Successfully migrated table [prov_vdc_logical_resource] : (35)
Migrating table [org_prov_vdc_template_bindings].
Successfully migrated table [org_prov_vdc_template_bindings] : (0)
Migrating table [vds_context].
Successfully migrated table [vds_context] : (0)
Migrating table [real_network].
Successfully migrated table [real_network] : (154)
Migrating table [vdc_compute_policy_alloc_type].
Successfully migrated table [vdc_compute_policy_alloc_type] : (0)
Migrating table [vm_container].
Successfully migrated table [vm_container] : (1,239)
Migrating table [vdc_network_profile].
Successfully migrated table [vdc_network_profile] : (0)
Migrating table [org_vdc_network_services].
Successfully migrated table [org_vdc_network_services] : (0)
Migrating table [org_vdc_nsxt_dfw].
Successfully migrated table [org_vdc_nsxt_dfw] : (0)
Migrating table [logical_disk].
Successfully migrated table [logical_disk] : (3)
Migrating table [vdc_logical_resource].
Successfully migrated table [vdc_logical_resource] : (640)
Migrating table [network_backing].
Successfully migrated table [network_backing] : (0)
Migrating table [logical_network].
Successfully migrated table [logical_network] : (2,446)
Migrating table [real_network_backing].
Successfully migrated table [real_network_backing] : (154)
Migrating table [vapp_vm].
Successfully migrated table [vapp_vm] : (1,421)
Migrating table [vapp_logical_resource].
Successfully migrated table [vapp_logical_resource] : (1,231)
Migrating table [vdc_storage_class_settings].
Successfully migrated table [vdc_storage_class_settings] : (0)
Migrating table [vdc_lr_encumbrance].
Successfully migrated table [vdc_lr_encumbrance] : (0)
Migrating table [shared_org_vdc_network].
Successfully migrated table [shared_org_vdc_network] : (10)
Migrating table [network_interface].
Successfully migrated table [network_interface] : (1,439)
Migrating table [logical_network_ip_scope].
Successfully migrated table [logical_network_ip_scope] : (2,584)
Migrating table [gateway_interface].
Successfully migrated table [gateway_interface] : (18)
Migrating table [vapp_vm_sclass_metrics].
Successfully migrated table [vapp_vm_sclass_metrics] : (1,421)
Migrating table [vapp_product_info].
Successfully migrated table [vapp_product_info] : (6)
Migrating table [vm_gosc_status].
Successfully migrated table [vm_gosc_status] : (1,421)
Migrating table [vapp_eula].
Successfully migrated table [vapp_eula] : (0)
Migrating table [vapp_vm_disk_storage_class].
Successfully migrated table [vapp_vm_disk_storage_class] : (20)
Migrating table [guest_personalization_info].
Successfully migrated table [guest_personalization_info] : (1,421)
Migrating table [address_translation].
Successfully migrated table [address_translation] : (0)
Migrating table [gateway_assigned_ip].
Successfully migrated table [gateway_assigned_ip] : (0)
Migrating table [vapp_property].
Successfully migrated table [vapp_property] : (46)

Successfully restarted sequence seq_config for table config.config_id at value 43,687
Successfully restarted sequence seq_vcenter_supported_builds for table vcenter_supported_builds.id at value 7
Successfully restarted sequence seq_audit_event_internal_id for table audit_event.internal_id at value 6,627,774
Successfully restarted sequence seq_cell_instance_id for table cells.instance_id at value 522
Successfully restarted sequence seq_os for table os.os_id at value 1,106
Successfully restarted sequence seq_guest_os_family for table guest_osfamily.family_id at value 4
Successfully restarted sequence seq_resource for table resource_item.resource_id at value 69,673
Successfully restarted sequence seq_disk_adapter_type for table disk_adapter_type.diskadapter_id at value 7
Database migration succeeded.
SUCCESS:329
FAILED:0
SKIPPED:0
To start using this database, run the cell-management-tool reconfigure-database command on each cell in your server group by executing the following command with the migrated databases connection properties:
/opt/vmware/vcloud-director/bin/cell-management-tool reconfigure-database

A common error you might receive here is shown below, if you receive this error then ensure your IP address has been properly added above to the pg_hba.conf file.
SEVERE: Connection error:
org.postgresql.util.PSQLException: FATAL: no pg_hba.conf entry for host "10.10.11.8", user "vcloud", database "vcloud", SSL off'

Integrate existing settings.

  1. On the migration source copy off the following files. Do not edit any entries in these files in this process. I used WinSCP to move the files between the two devices. 
    /opt/vmware/vcloud-director/certificates.ks
/opt/vmware/vcloud-director/etc/certificates
/opt/vmware/vcloud-director/etc/global.properties
/opt/vmware/vcloud-director/etc/proxycertificates
/opt/vmware/vcloud-director/etc/responses.properties
/opt/vmware/vcloud-director/etc/truststore
  2. We will temporarily upload all files to a temporary directory /root/copied on the new appliance. 
    mkdir /root/copied
  3. Rename the file certificates.ks to certificates.ks.migrated We will not be using the migrated certificates.ks.migrated file just yet. Do not try to replace the existing file on the new appliance.
  4. Copy all of the files to /root/copied directory.
  5. Log into the new appliance and ensure vcloud user and group are properly set on the new files we just copied over. 
    chown vcloud:vcloud /root/copied/*
  6. On the new appliance you will want to rename some of the original files to keep them for reference. 
    cd /opt/vmware/vcloud-director/etc/
mv certificates certificates.original
mv global.properties global.properties.original
mv proxycertificates proxycertificates.original
mv responses.properties responses.properties.original
mv truststore truststore.original
  7. On the new appliance copy the files from /root/copied to their respective directories. 
    mv /root/copied/certificates.ks.migrated /opt/vmware/vcloud-director/
mv /root/copied/certificates /opt/vmware/vcloud-director/etc/
mv /root/copied/global.properties /opt/vmware/vcloud-director/etc/
mv /root/copied/proxycertificates /opt/vmware/vcloud-director/etc/
mv /root/copied/responses.properties /opt/vmware/vcloud-director/etc/
mv /root/copied/truststore /opt/vmware/vcloud-director/etc/

Starting up vCloud Director 9.7 appliance to confirm integrity.

  1. On the new appliance run the command to reconfigure the vCloud Director service. 
    /opt/vmware/vcloud-director/bin/configure --unattended-installation --database-type postgres --database-user vcloud --database-password db_password_new_primary --database-host eth1_ip_new_primary --database-port 5432 --database-name vcloud --database-ssl true --uuid --keystore /opt/vmware/vcloud-director/etc/certificates.ks --keystore-password root_password_new_primary --primary-ip appliance_eth0_ip --console-proxy-ip appliance_eth0_ip --console-proxy-port-https 8443

    Where:

    • The –keystore-password value matches the initial root password of this appliance.
    • The –database-password value matches the database password that you set during the appliance deployment.
    • The –database-host value matches the eth1 network IP address of the primary appliance.
    • The –keystore value is the path to the certificates.ks file you backed up in Step 10.b.
    • The –primary-ip value matches the eth0 network IP address of the appliance.s
    • The –console-proxy-ip value matches the eth0 network IP address of the appliance.
    • The above is directly quoted from VMware’s documentation found here [Migrating vCloud Director with an External Microsoft SQL Database to vCloud Director Appliance].

You can abbreviate much of the above line with shorter commands and even exclude some of the parameters. Here is the commands that I used to successfully do the process.

/opt/vmware/vcloud-director/bin/configure --unattended-installation \
-dbtype postgres \
-dbhost 10.10.11.11 \
-dbport 5432 \
-dbname vcloud \
-dbuser vcloud \
-dbpassword 'VMware!1' \
-g \
-k /opt/vmware/vcloud-director/certificates.ks \
-w 'VMware!1' \
-ip 10.10.10.10 \
-cons 10.10.10.10 \
--console-proxy-port-https 8443
  1. Run the command to start the vCloud Director service. Once it is back online you will want to confirm that everything has worked correctly to this point, your VCD is populated, etc. 
    service vmware-vcd start

    NOTE You can monitor the progress of the cell startup at /opt/vmware/vcloud-director/logs/cell.log.

  2. Delete the disconnected cells that belonged to the old environment. Simply go to the manage and monitor tab, click the cloud cells, right click and select delete.
  3. If you find errors and you need to utilize the old flex.ui then you can enable it with the following command. This is no longer an option in Cloud Director 10.1.1 
    /opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n flex.ui.enabled -v sys-admin-only

Upgrade from 9.7 to 10.1.1

  1. I recommend creating a snapshot and clone of your successfully running vCloud Director 9.7 appliance before proceeding.
  2. Download vCloud Director gz file “VMware Cloud Director 10.1.1-Virtual Appliance update package” from VMware. This file will be in the following naming convention: VMware_vCloud_Director_v.v.v.v-nnnnnnnn_update.tar.gz
  3. If you have multiple cells then use your favorite browser log into your VAMI interface at https://appliance_ip_address:5480 to determine which cell is the current primary, you need to use the primary appliance when you do the database backup later in this process.
  4. Download the update package and create an iso out of the file and mount it as a CDROM.
  5. Log into the appliance and insure the directory /mnt/cdrom exists, if not create it by running: mkdir /mnt/cdrom
  6. Mount the CDROM. 
    mount /dev/cdrom /mnt/cdrom
  7. Create a new director to hold the update content. 
    mkdir /tmp/local-update-package
  8. Extract the package in the newly created directory. 
    tar -zxf /mnt/cdrom/VMware_Cloud_Director_10.1.1.4916-16288798_update.tar.gz  -C /tmp/local-update-package
  9. We now need to configure the update repository to use this directory. 
    vamicli	update --repo file:///tmp/local-update-package
  10. Test to ensure the update directory is being read correctly. If it is then you should be presented with the release version. 
    vamicli update --check
  11. Check the cell status to ensure no jobs are running. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
  12. Quiesce the cell to prevent new jobs from running and complete existing jobs. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell -q true
  13. Shutdown vCloud Director management cell. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
/opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –shutdown

    NOTE If there are queued jobs then you can override the wait by running: service vmware-vcd stop

  14. Apply the upgrade to the cell. 
    vamicli update --install latest
  15. If you are running multiple cells then repeat the above steps on all additional cells.
  16. On the primary appliance we noted above run the DB backup process. 
    /opt/vmware/appliance/bin/create-db-backup
  17. Now lets upgrade the Cloud Director database utility. 
    /opt/vmware/vcloud-director/bin/upgrade
  18. When prompted with “Do you wish to upgrade the product now? [Y/N]” enter Y
  19. When prompted with “Enter [Y] after the backup is complete.” enter Y
  20. When prompted with “Start it now? [y/n]” enter N
  21. Reboot the Cloud Director appliance. 
    shutdown -r now
  22. Once the appliance is done rebooting log into your web interface and ensure everything is working as expected. NOTE If you run into a problem not being able to access /provider URL then you might have to ensure the URL you are supplying the browser is allowed. Cloud Director 10 is explicit in the URLs that it will accept as source. You can look at this article here if you need to modify your URLs called Troubleshoot Failed Access to the vCloud Director User Interface.

Update Certificates.

  1. If you have not already done so then upload the original file from the old VCD cell to /opt/vmware/vcloud-director/certificates.ks.migrate
  2. Rename the original certificates.ks file. 
    mv /opt/vmware/vcloud-director/certificates.ks /opt/vmware/vcloud-director/certificates.ks.original
  3. Rename the migrated certificates.ks file. 
    mv /opt/vmware/vcloud-director/certificates.ks.migrated /opt/vmware/vcloud-director/certificates.ks
  4. Check the cell status to ensure no jobs are running. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
  5. Quiesce the cell to prevent new jobs from running and complete existing jobs. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell -q true
  6. Shutdown vCloud Director management cell. 
    /opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –status
/opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell –shutdown

    NOTE If there are queued jobs then you can override the wait by running: service vmware-vcd stop

  7. Run the configuration tool to import the new certificate. 
    /opt/vmware/vcloud-director/bin/configure
  8. If asked “Please enter the path to the Java keystore containing your SSL certificates and private keys:” enter the location you uploaded the file to. If our case: /opt/vmware/vcloud-director/certificates.ks

    NOTE If you receive an error the file cannot be found then ensure the path is entered correctly. If you receive an error saying unable to read file, please correct the file permissions then you will need to change the ownership to vCloud by running: chown vcloud:vcloud /opt/vmware/vcloud-director/certificates.ks

  9. Enter the keystore password. This is a unique password that was given to the original certificates.ks keystore on the source machine.
  10. Once provided Cloud Director will show connecting to the database, confirm the DB creds were read successfully, and show a DB configuration complete message.
  11. When asked if you want to start it now, enter Y
  12. Confirm that Cloud Director 10.1.1 is now working as expected. If you get the error “One or more VC/NSXs have been detected to be disconnected. This could be due to SSL verification failures.” then you can reference the article as mentioned in the error titled vCenter Server and/or NSX are disconnected after a VCD 10.1 upgrade (78885)

Reference documentation:


Upgrading vCloud Director and Patching the vCloud Director Appliance
Migrating to vCloud Director Appliance
Migrating vCloud Director with an External Microsoft SQL Database to vCloud Director Appliance
Configure External Access to the vCloud Director Database
Deploying the vCloud Director Appliance
Prerequisites for Deploying the vCloud Director Appliance
Preparing the Transfer Server Storage
Upgrade the vCloud Director Appliance
Create and Import CA-Signed SSL Certificates to the vCloud Director Appliance
How to setup an NFS Server on CentOS 6