A Confused Programmer's Blog

Recently I was setting up environment with WAN emulation on ESX VMs. The basic setup contained Windows XP/7 VM connected to Windows server VM with WAN emulator in the middle (Ubuntu VM). Traffic sent from XP machine utilized the bandwidth set by the WAN emulator; however, traffic sent from Windows 7/2008 was really struggling to rise above 15KB/sec.

After looking at sniffer’s traces I discovered that while XP client was continuously sending packets of 1514 bytes length:

VMs with more advanced OS were trying to optimize performance by increasing packet size:

The larger packets were dropped (never arrived to the destination), sender waited 300ms for timeout and started again with smaller packets until deciding again to go up with sizes (as the trace above shows) and repeating the vicious cycle. After some research I discovered that the feature responsible for this optimization is “Large Send Upload” option that is supported by more advanced NICs, which explains why XP VMs didn’t try this optimization. Here’s how we can disable it:

There’s also a programmatic solution to disable this feature on-the-fly. It goes something like this:

public static bool DisableLargeUploadOffload()
{
 RegistryKey networkAdapters = Registry.LocalMachine.OpenSubKey(@"System\CurrentControlSet\Control\CLASS\{4D36E972-E325-11CE-BFC1-08002BE10318}");
 const string largeUploadOffloadKey = "*LsoV1IPv4";
 bool hasUpdatedAnything = false;
 foreach (var adapterNum in networkAdapters.GetSubKeyNames())
 {
  if (!adapterNum.StartsWith("0")) continue;
  RegistryKey concreteAdapter = networkAdapters.OpenSubKey(adapterNum, true);
  if (concreteAdapter == null) continue;
  if (concreteAdapter.GetValueNames().Contains(largeUploadOffloadKey))
  {
   Console.WriteLine("Disabling large upload TCP offload for {0}", concreteAdapter);
   concreteAdapter.SetValue(largeUploadOffloadKey, "0");
   hasUpdatedAnything = true;
  }
 }
 return hasUpdatedAnything;
}


After the registry is updated, it is necessary to restart network adapters – either by simply rebooting the VMs or by using WMI (check out Win32_NetworkAdapter table’s Enable() and Disable() methods).