Older versions of iperf3 cannot properly measure networking throughput for Windows

One of the things I test when reviewing mini PCs for Liliputing is WiFi performance or network throughput. And up until recently I’ve used the popular iperf3 tool to do that, since it allows me to see how real-world performance compares with the speeds you’d expect from the specifications provided by WiFi chip makers.

For some time though, I’ve noticed that WiFi speeds have been much higher when I ran the test on systems running Ubuntu Linux than they were when running the same test. Now Microsoft has provided an explanation for the discrepancy in an article entitled Three Reasons Why You Should Not Use iPerf3 on Windows .

Those three reasons boil down to:

1. ESnet, the team that develops iPerf3 doesn’t officially support Windows.
2. While there are versions of the tool that have been compiled for Windows, they do not make native API calls on Windows, and instead use Cygwin as an emulation layer, which could add some overhead.
3. Up until recently, there  was a Windows build available for download from the iperf3 website , but it was nearly 8 years old and contained a bug in cygwin that causes “poor performance on high speed-high latency and high bandwidth networks.”

At first, I thought Microsoft’s second point, namely that “iPerf3 is Emulated on Windows,” appeared to be the most likely cause for the lower than expected throughput I’ve encountered.

So I installed Microsoft’s suggested alternative of ntttcp , as there are both Windows and Linux versions of the tool, and because it supports testing between Windows and Linux. That second point is critical, given that I always test throughput to and from a Linux server regardless of client OS.

With the “ntttcp” tool installed on both Windows and Ubuntu on a mini PC client, and “ntttcp” installed on the Ubuntu server, I measured the throughput on each OS over both Ethernet and WiFi. And this time the Windows and Ubuntu results were much closer.

However I then decided to investigate Microsoft’s third point. I had been downloading the Windows build of iperf3 from the iperf website, which means I was running iperf3 3.1.3 rather than the latest version, which is iperf3 3.16.

After Microsoft published its article though, ESnet removed that download link, suggesting there was something to Microsoft’s assertion that the old build of the software didn’t perform properly on Windows.

So I also installed a Windows version of iperf3 3.16 and ran the test again on Windows. In the table below, for Ethernet I used a connection between two 2.5 Gbps ports. , and for WiFi, I connected over networks on 2.4, 5.0 and 6.0 GHz bands.

While throughput measured with iperf3 3.1.3 on Windows was significantly lower than the measurements I saw with Ubuntu, the Windows ntttcp and iperf3 3.16 throughput measurements were much closer. And there is very little difference in measured throughput compared with nttcp, with variances between 0% and 3.6%, i.e., likely within the testing margin of error.

The muted emulation performance penalty appears to be minimal, and is likely not detectable given the normal variations of throughput encountered while testing WiFi (it is not an exact science when tested outside of lab environments).

Now that it’s clear that the discrepancy between WiFi network throughput that I was seeing while reviewing mini PCs was due to the measurement tools and not the operating systems themselves, I’ll either make sure to use the latest tools or go back to simply reporting the Ubuntu-only results since they should be similar to what you’d see with Windows.

via Slashdot

Update: This article was first published April 23, 2024 and most recently updated April 24, 2024 with a comparison between throughput measured by iperf3 3.1.3 and iperf 3.16 on Windows. The article originally only compared iperf3 3.1.3 to ntttcp.