Power,Speed, Performance: Memorial Day Motorsports & NSX

Before we start this blog I would like to introduce myself my name is shreya and I am the senior instructor at Getprotech. Getprotech providing VMware training in delhi. Let’s back to the topic. With Memorial Day weekend coming up, for me, it’s all about hamburgers, hot dogs, and fast car racing. I am huge Formula 1 addict, but Memorial Day is a windfall of racing from the F1 Monaco Grand Prix, to NASCAR’s Coke 600, and certainly the Indianapolis 520 all on the same day! The raw speed and performance of these races reminisce me of a 2016 VMworld presentation on NSX performance.

The advocacy still comes up now and repeatedly that "hardware is faster than software." Network guys like me or some other guys just assume that’s true. So, it came as amazement to me when I saw the session which turned that imagery on its head. In this session, the presenter displayed that software is faster than hardware, way faster. Of course, I was doubtful at first but speedily learned that virtual networking and physical networking is like the difference between the race car and the pace car. I always thought the physical switch was the race car, but in the throughput demonstration, Samuel showed two Virtual machines running on the same ESXI host with switching, NSX routing, and firewalling between them could get up to 107G! This information amazed me. Sort of like the same experience I had when I saw my first race and 41 cars took the green flag at 180 mph. The demonstration went further to demonstrate throughput between two different ESXI hosts connected by a physical switch was 77G. What caught my mind is that the physical switch was now the bottleneck liken to the NSX vswitch. Like the restrictor plate NASCAR mandates in the Daytona cars to slow them down, I understood software could be faster than hardware. So, what this means is by running NSX a client can not only receive line rate performance but possibly better performance as compared to their physical switch uplinks which today are usually numerous of 10G. You may have seen vrealize network insight product show routed traffic as a percentage of the total flows. What this means is the switch is performing that routing function at ESXI host uplink speeds (~12G). By deploying NSX, that same traffic could run up to 200G or a 15x improvement.

Full disclosure, this is a nuanced discussion and most application communication is across ESXI hosts, but it is rapid nonetheless. There are some optimizations that vNICs and pNICs leverage to deliver the performance numbers I specified above. LRO, TSO and RSS are all optimizations required to convey on those numbers. However, these optimizations are same in newer operating systems and newer pNICs. The interesting difference between virtual and physical is how packets are segmented. You may be cognizant with MTU. MTU is the size of the biggest network layer PDU (protocol data unit) transmitted in a singlular network transaction. You may have noted presenters of NSX say we need a minimum 1700 byte MTU. Ethernet defaults are 1500, but switches can go up to 10K. The greater MTU brings greater efficiency because every network packet brings more user data and the greater efficiency means an development in bulk protocol throughput. Within a ESXI, MTU is irrelevant with few of these optimizations as all the connections between virtual machines is done in memory and not fragmented. So, a more important maximum is maximum segment size. Since we skip fragmenting packets down to MTU sized packets, we could work with segments as large as 66K thanks to LRO and TSO within the ESXI host. Even across ESXI hosts, a NIC card that supports LRO for VXLAN helps press the throughput up tremendously. We play by a different set of rules in the ESXI host. Call it fraud, call it less tire pressure, call it an lawless spoiler, call it what you can say, but the VMware NSX race car is very competing in the game of speed.

This article is brought to you by Getprotech. Getprotech provides the VMware training in delhi