Network Router and Adapter Testing Methodology

Before we dive right into the router and adapter performance, we need to discuss a testing methodology.  Being that this is one of the first reviews I’ve done for networking gear, I needed to set up my own ‘test bed’ for routers and network adapters.  Thanks to the great people over at Small Net Builder and their excellent “How to Test a Wireless Router” article, I had some good foundations to build from so I could get accurate and consistent results across multiple tests.

To take as many variables out of the equation and maintain consistency across multiple tests I picked up two identical Dell Optiplex machines off of eBay to use as my “WAN” and “LAN” networks.  The machine specs for the two machines are:

Dell Optiplex 745 Desktop PC

These are both hooked up to a basic Dell keyboard, mouse and 20” LCD using a Syba 2-Port USB KVM Switch.  When needed, I used 2 foot CAT 6 network cables from monoprice.com.

27-test-bed-setup.jpg

The Optiplex machines certainly aren’t the most powerful machines around, but to test network connectivity you really don’t need the Dream System from the PCPer Leaderboard.  I just wanted a pair of machines that were identical to ensure consistent results from test to test.

To give us a baseline for both wired and wireless connectivity, we’re using one of the better routers out there, the Apple Airport Extreme.  This little box from Apple has Gigabit WAN/LAN ports, 802.11 a/b/g/n wireless and supposedly ‘easy’ setup.  I say supposedly because I had to fight with the Airport software to do some of the ‘advanced’ configuration I wanted to do such as setting up different subnets on the LAN/WAN ports, but that’s a story for another day.

28-apple-airport-stock.jpg

For network adapter testing, there’s an open PCI Express slot and some USB ports in each of the Optiplex machines that are available.  Since most USB adapters can also easily be used on a laptop, I’ll do testing of USB adapters using a laptop.  The added benefit of using a laptop for USB adapter testing is that I’ll be able to test from a few other locations to simulate some real use test cases.  For this testing I’ve drummed my old MSI laptop back into service.

MSI EX625-227US Laptop

The MSI EX625 originally came with a general built in 802.11 b/g/n adapter, but I upgraded the internal Wi-Fi adapter to the Centrino Ultimate N as it’s one of the better like one of the better mini PCI Express adapter on the market.  With ‘3×3’ triple streams and advertised speeds up to 450 Mbps it’ll be perfect to use as a baseline to test router Wi-Fi speeds as well as to compare to other Wi-Fi adapters.

29-msi-ex625.jpg

Though hardware is only half the equation.  We still need a way to actually measure network throughput and other stats of the routers and adapters so we can compare and contrast them against each other with some sort of consistency.  There are simple online speed, but there’s just too much variance if we use that.  For consistency sake, the machines will not be connected to the Internet at all and only networked amongst themselves.  We’ll then use three pieces of software wrapped up in a script to give us a good picture of network performance.

First up is a simple DOS based tool contributed by Matrix21 over on Smallnetbuilder.com, aptly named, “Max_session_tool.”  The Max Session Tool will open up UDP connections between two hosts until the receiving end starts to drop the packets giving us the maximum number of network sessions that the endpoints and network hardware can handle.  Since the router is the primary keeper of the sessions, we’ll only run this test for routers. 

30-max-sessions-tool.jpg

Next up is the simple ping test.  Most people will be familiar with using the ping command to check connections between two machines.  With this simple tool we can check the minimum, maximum and average time needed to send a packet of data between two machines, as well as give us a total number of dropped packets.  The ping test will give us a good way to check for latency or for any other network issues.

31-ping-tool.jpg

Finally are a pair of great tools from Pete over at Totusoft.   We’re using LAN Speed Test (v3) paired with the LST Server software.  While there is a free ‘Lite’ version of the LAN Speed Test available, at only $5 each, both LAN Speed Test and LST server are worth every penny and then some.  LAN Speed Test will run a battery of file upload/downloads against a file share on another machine to give you Average, Minimum and Maximum Upload/Download speeds.  Adding in the LST server let’s you upload/download the data into the target system’s memory as opposed to hard disk to remove any impact hard disk speeds/cache would have on the actual speed values.  I can’t thank Pete enough as he happily answered some of my questions and even took some suggestions I had and added them into the applications.  For anyone looking to do any speed testing, I highly suggest checking out Totusoft’s offerings.

32-lan-speed-tests.jpg

With these three tools we can get a pretty good snapshot of network connectivity and throughput.  When I first started working through the tests, it was a lot of manual work, so make life easier, I strung together a batch file to automate each set of tests. 

33-network-testing-batch-1.jpg

The batch file does the following:

  • For routers, run the Max Sessions Tool 7 separate times and document the maximum sessions allowed.
  • Run 7 tests of 100 pings from client to target computer and document the Minimum, Maximum, Average ping time as well as any dropped packets.
  • Run 7 upload and download tests using LAN Speed Test with 10 MB files, 10 times for each file.  Document Average, Minimum and Maximum Upload and Download speeds in Mbps.

34-network-testing-batch-2.jpg

Once we have all the data, it’s added to a master spreadsheet.  To try to take any wild values out of the mix, we drop the highest and lowest values from the test results and then average the remaining ones for our ‘final’ test value for each metric.

35-test-spreadsheet.jpg

To add one final piece to the puzzle, we will test Wi-Fi connectivity from 3 locations that we will call for simplicity’s sake, ‘Lab Test’, ‘Real World Test’, and ‘Torture Test’.  In the ‘Lab Test’ I’ve set up both the router and wi-fi adapter in my office and approximately 3 feet from each other.  For the second ‘Real World Test’ the signal will have to travel half way across the house through two walls and one floor into the Living room about 45 feet away.  Finally, we’ll wrap things up with a ‘Torture Test’ where the wireless signal has to travel 70 feet into my Laundry Room and punch through four walls, a floor, and everything in between.  I don’t know what the builders lined the walls of the Laundry room with, but not only does it lay the smack down on wireless signals, but I think I can use it as a nuclear bunker.

36-house-test-setups.jpg

So, without further ado, let’s dive into some actual results from our wired and wireless tests.

« PreviousNext »