Network Router and Adapter Testing Methodology
Before we dive right into the router and adapter performance, let’s discuss our testing methodology again. As mentioned in earlier reviews we’ve worked to take as many variables out of the equation and maintain consistency across multiple tests. We’re using two identical Dell Optiplex machines as the “WAN” and “LAN” networks with the following specifications:
- Core 2 Duo E6600 CPU (2.4 GHz)
- 2 GB (2 x 1 GB) Kingston HyperX DDR2 800 (PC2 6400) SDRAM
- Western Digital WD1600AAJS 160 GB HD (8 MB Cache/7200 RPM)
- Trendnet TEG-PCITXRL Low Profile Gigabit PCI Adapter
Both Dell machines are hooked up to a keyboard, mouse and 20” LCD using a Syba 2-Port USB KVM Switch. When needed, we use 2 foot CAT 6 network cables from monoprice.com.
As a baseline to compare both wired and wireless connectivity, we’re using one of the better routers out there, the Fifth Generation Apple Airport Extreme Base Station (A1408). This little box from Apple has Gigabit WAN (1) and LAN (3) ports as well as 3×3:3 simultaneous dual band 802.11 a/b/g/n wireless that should theoretically be able to achieve speeds of up to 450 Mbps. Though Apple, unlike most other vendors, is smart enough not to tout this number, as it’s unlikely end users will be able to achieve it.
To test PCI Express network adapters, we do have an open PCI Express slot in each of the Optiplex machines that are available. For USB based adapters, we’ll do testing with a laptop. Using a laptop for USB adapter testing will give us the option to not only test from our “Lab” where the Optiplex machines are located, but will allow us to test from other locations to simulate some real use test cases as well. For this testing I’m using an MSI laptop with the following specifications:
- Core 2 Duo P7350 (2.0 GHz) CPU
- 4 GB (2 x 2 GB) Corsair DDR2/800 SODIMM RAM
- Kingston SSDNow V+ 100 96 GB SSD (SVP100S2/96G)
- ATI Mobility Radeon HD4670 (512 MB VRam)
- Intel Centrino Ultimate N 802.11n (draft) Wi-Fi Adapter (633ANHMW)
The MSI EX625 came with a built in 802.11 b/g/n adapter, but we’ve upgraded the internal Wi-Fi adapter to the Centrino Ultimate N as it’s one of the better like one of the better mini PCI Express adapters on the market that might be found in a laptop. With 3×3:3 triple streams and advertised speeds up to 450 Mbps it’ll be perfect to use as a baseline to test router Wi-Fi speeds as well comparisons to other USB based Wi-Fi adapters.
Hardware is only half the equation though and we still need a way to actually measure network throughput and other stats of the routers and adapters so we can compare and contrast them against each other with some sort of consistency. There are simple online speed tests, but there’s just too much variance if we use that. For consistency sake, the machines will not be connected to the Internet at all and only networked with the test routers and each other. We’ll then use three pieces of software that I’ve wrapped up in a script to give us a good overall picture of network performance.
The first tool we use is a simple DOS based application contributed by Matrix21 over on Smallnetbuilder.com, aptly named, “Max_session_tool.” The Max Session Tool will open up UDP connections between two hosts until the receiving end starts to drop the packets. This will tell us the maximum number of network sessions that can be maintained between the LAN and WAN machine ang give us an idea of how well the router can handle multiple sessions. Since this test mainly tests a router’s capability to open and maintain sessions, we will only run this test for the wired tests through a router.
Next up is the ever useful ping test. Anyone that’s done even basic network troubleshooting will be familiar with the ping command that is used to check connections between two machines. This simple tool will test the reachability of a host on the network and measure rout trip time for messages sent from the host to the destination machine. With this simple tool we can gather tthe minimum, maximum and average time needed to send a packet of data between machines, as well as let us know if there are any dropped packets.
Our final tool is actually a a pair of great tools from Pete over at Totusoft. Using Totusoft’s LAN Speed Test (v3) paired with the LST Server software we can get data throughput speeds between two machines. With LAN Speed Test, you can run file upload/downloads against a file share on another machine to give you Average, Minimum and Maximum Upload/Download speeds. However, adding LST Server into the mix lets you upload/download the data into the target system’s memory as opposed to hard disk. This way you remove any impact the hard disk speeds/cache would have on the actual speed values. While there is a free ‘Lite’ version of the LAN Speed Test available, at only $5 each, both LAN Speed Test and LST server are worth every penny and then some.
With these three tools we can get a pretty good snapshot of network connectivity and throughput. Initially I was manually running each test after each other, but as there’s such a large amount of tests that needs to be run, I strung together a batch file to automate each set of tests.
The batch file does the following:
- For routers, it will run the Max Sessions Tool 7 times and document the maximum sessions between the LAN and WAN machine for each run.
- It will then run 7 tests of 100 pings each from the LAN to the WAN computer and document the Minimum, Maximum and Average ping time as well as any dropped packets.
- Finally it will run 7 upload and download tests between the LAN and WAN using LAN Speed Test and LST Server. The LAN speed test will upload and download a 10 MB file 10 times for each run and then document Average, Minimum and Maximum Speeds in Mbps for both uploading and downloading.
Once all the tests have run, we transfer the data from the batch file logs over to a master spreadsheet. To minimize the impact of any extreme test results we drop the highest and lowest values for each test and then average the remaining ones for our ‘final’ test value for each metric.
We do have a few extra test procedures involved for wireless testing. We will always test both 5 GHz and 2.4 GHz networks separately if they are available. To ensure wireless testing is as consistent as possible across all our tests, we will always attempt to use the following Wireless networking settings on routers/adapters.
- Security: WPA2 Personal / AES
- 2.4 GHz Channel: 11
- 5 GHz Channel: 157
- Use Wide Channels/Channel Bandwidth: 40 Mhz
- Multicast Rate: Low
If a wireless adapter has more than one mode, say a “Standard” and a “High Power” mode, we’ll also run all the tests for each mode to see if there is any benefit to using one mode over the other.
Finally, since most people are not going to be sitting right next to their router when using USB based wireless adapters and a laptop, we will run all of our wireless tests in three different locations. For simplicity’s sake we are calling them the ‘Lab Test’, the ‘Normal Use Test’, and the ‘Torture Test’. In the ‘Lab Test’ I’ve set up both the router and Wi-Fi adapter in my office approximately 3 feet from each other. For the second ‘Normal Use Test’ the signal will have to travel half way across the house through two walls and one floor into the Living room about 45 feet away. Finally, we’ll wrap things up with a ‘Torture Test’ where the wireless signal has to travel 70 feet into my Laundry Room and punch through four walls, a floor, and everything in between. I don’t know what the builders lined the walls of the Laundry room with, but not only does it lay the smack down on most wireless signals, but I think I can use it as a nuclear bunker.
With that said, let’s dive into the actual testing and see how well everything performed!
seems okay but that AP and
seems okay but that AP and wifi adapter is really overpriced. Wanted to also add that it is becoming more common for many upper end consumer level routers to have 600mw transmit power. Before I purchase, I usually look at the FCCID and check for it’s transmit power and if it does not have external antennas, then I also check for U.fl connectors. (replacing a crappy internal antenna on a high end router from linksys or netgear with a quality 5-7dbi antenna will significantly improve your coverage and performance.
PS if you have a single level home, a 9dbi rp-sma omni directional = awesomeness.
While I haven’t used this
While I haven’t used this particular device I’ve recently had to communicate with their technical support department regarding another device and I was impressed with the quality of their North American support department. The support representatives seemed to really care about my success with their equipment so I figured I’d throw that in there in the event its meaningful in considering their products.