VMware HCX: What happens during the network cutover of L2 Extension? – Part 2

Part 1 of the series covered first three steps in the flowchart below

Part 2 (this post) covers the final step of network cutover. We will continue from Step 4 here.

Step 4: Network Unextend or Cutover

Unextend the network using HCX portal by selecting the stretched networks that need to be cutover.

As soon as the network unextending task completes, the network i.e. default gateway is attached to the cloud gateway.

It is important to check the box to ‘Connect cloud network to cloud edge gateway’, as this is what will activate the network on the cloud side. If the box is not checked, the default gateway remains on-prem leaving the cloud network disconnected. There are situations where this box should be left unchecked, especially when not cutting over – more on that later.

The default behavior of the cloud gateway is to advertise the networks out by default, and hence these networks will now be learnt by on-prem via BGP. Just before or during the cutover, the on-prem networks can be disconnected or deleted from the on-prem router. Below example is showing B2 Router (NSX Edge in our case) but this could be any device with the network definitions.

BGP filtering in B1 can also be updated before cutover to allow these networks to be advertised by the cloud side into the on-prem.

R2#show ip prefix-list
ip prefix-list R1-DENY: 5 entries
   seq 5 permit 192.168.128.0/19 le 24
   seq 10 permit 192.168.160.0/19 le 24
   seq 15 permit 192.168.19.0/24
   seq 20 permit 192.168.48.0/24
   seq 25 permit 192.168.49.0/24
ip prefix-list R1-PERMIT: 1 entries
   seq 5 permit 192.168.0.0/16 le 32

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#no ip prefix-list R1-DENY seq 15 permit 192.168.19.0/24
R2(config)#no ip prefix-list R1-DENY seq 25 permit 192.168.49.0/24

R2#show ip prefix-list
ip prefix-list R1-DENY: 3 entries #deny list is now updated to allow the two netowrks
   seq 5 permit 192.168.128.0/19 le 24
   seq 10 permit 192.168.160.0/19 le 24
   seq 20 permit 192.168.48.0/24
ip prefix-list R1-PERMIT: 1 entries
   seq 5 permit 192.168.0.0/16 le 32

Network topology here again for quick reference. Refer part 1 for the original route tables before cutover or for additional details.

B1 Route table after cutover below. Voila, the migrated networks next hop is now cloud! Only 192.168.48.0/24 remains on-prem – probably waiting to be scheduled in the next wave.

The on-prem router (B2) now learns the networks from the cloud side via BGP (‘B’) and are not locally connected networks (‘C’) anymore!

In the above tests the connectivity resumed instantly within a handful of seconds. The route propagation is mostly instant within seconds in customer environments, and I have not come across big delays. Once everything is working as expected, the corresponding virtual port groups and remnants of the networks within the on-prem environment can be removed. The considerations shown here mostly cover BGP filtering, please leave in comments if you would like other on-prem scenarios covered.

Every environment is different! Ensure to test the end to end process to learn environment specific behavior and needs. The whole process can be easily tested using test network/workloads.

Security update

Security configuration updates may NOT be needed in cases where the traffic flows between the private segments are already allowed in the existing security rules.

Taking the below screenshot example, if the ‘cloud-subnets’ group already cover the migrated subnets (say using a supernet or dynamic grouping), there is no further action required as the migrated VMs are automatically part of the subnet and continue to access the required services. Firewalls could potentially be filtering at S1 and S2 in the layout diagram above.

If the migrated subnets or workloads are not part of the security rules, they will have to be added to the relevant groups (‘cloud-subnets’ in the above example or new groups). The rules can be set and tested even before the migrations are started.

Hopefully the considerations explained above are useful. Please feel free to reach out if you would like to cover other scenarios in the future blog posts. Do leave comments and feedback, I already know what Chandler would say –

Thanks for reading.