Gigabit Internet Service - Are you getting what you're paying for?

Post date: May 28, 2019 10:31:45 PM

I was recently been approached by a Centurylink / AT&T (DirecTV) salesman about my internet and cable television service. What caught my attention about the pitch, is that the offer for services was significantly lower price than I was / am currently paying. I signed up for service. I've been through this before w/ Centurylink when fiber was first installed in my neighborhood so I knew better than to immediately cancel service from my current provider (Cox). Last time I did this, the techs installing the service were still learning the ropes, and there weren't enough of them. That meant that when my service broke, it took too many days to fix it and I considered that unacceptable and dropped their service. This time, I was assured that the service would be better, that they've updated their networks since then and they have enough knowledgeable field technicians to service the area.

Anybody paying a premium for gigabit internet service should be doing what I did. Don't just take their word for it, check that you get what you paid for. Luckily, there's online tools readily available to check your download and upload speeds. I used the well known speedtest.net, but I used command line version so I could run this on a Linux box connected to BOTH services at the same time. This allowed me to do a completely fair comparison of the two internet service providers, from the same hardware, same time of day, to the same speed test endpoints, etc. The first thing you need to know about speedtest.net if you are checking out your speeds from Cox or Centurylink, is that by default it picks the respective speedtest server on THEIR network. So, if you are on Cox, you get Cox's speedtest server, and if you are on Centurylink, you get Centurylink's speedtest server. This is great if you are checking to see if your internet connection has problems between your house and your ISP, but isn't a full representation of speeds from you to the internet. Here is the results of my tests from each service provider to their speedtest servers:

Retrieving speedtest.net configuration...

Testing from Cox Communications (xx.xx.xx.xx)...

Retrieving speedtest.net server list...

Selecting best server based on ping...

Hosted by Cox - Omaha (Omaha, NE) [9.71 km]: 9.528 ms

Testing download speed..........................................................

Download: 754.29 Mbit/s

Testing upload speed............................................................

Upload: 37.33 Mbit/s

###

Retrieving speedtest.net configuration...

Testing from CenturyLink (xx.xx.xx.xx)...

Retrieving speedtest.net server list...

Selecting best server based on ping...

Hosted by CenturyLink (Omaha, NE) [7.19 km]: 5.498 ms

Testing download speed..........................................................

Download: 875.52 Mbit/s Testing upload speed............................................................

Upload: 472.43 Mbit/s

###

You can see that each has comparable download speeds, with Centurylink being a little faster (I assume due to lower latency connection), and Cox upload speed being much lower which is expected with the way they split bandwidth on DOCSIS 3.1. All these tests mean though, is that I don't have any issues with my system, cables, router, or connection to either ISP. I'm not paying for access to my ISP's network though am I? I'm paying for access to the internet, so let's test speeds to a speed test server on the internet. Here's the result when I pick a server on the internet, but still close to me.

Retrieving speedtest.net configuration...

Testing from Cox Communications (xx.xx.xx.xx)...

Retrieving speedtest.net server list...

Selecting best server based on ping...

Hosted by Binary.Net (Omaha, NE) [9.71 km]: 26.91 ms

Testing download speed..........................................................

Download: 662.99 Mbit/s

Testing upload speed............................................................

Upload: 36.95 Mbit/s

###

Retrieving speedtest.net configuration...

Testing from CenturyLink (xx.xx.xx.xx)...

Retrieving speedtest.net server list...

Selecting best server based on ping...

Hosted by Binary.Net (Omaha, NE) [7.19 km]: 32.44 ms

Testing download speed..........................................................

Download: 215.93 Mbit/s

Testing upload speed............................................................

Upload: 174.07 Mbit/s

###

Now Cox is way ahead in speeds, and Centurylink download was 1/3 the speed of Cox. What doesn't show above is that during the speedtest run for Centurylink, the dots indicating download seemed to stall several times. In other words the download seemed to come in at bursts. I wanted to find out exactly why this is, so I did a packet capture of a speedtest run over the Centurylink line and took a look at it with Wireshark. Here's what I found:

You can see in the expert information from Wireshark, that it detected 80 TCP retransmits. Those are from packets being lost on the way from me to the speedtest server, and needing to be re-transmitted. Those lost packets are also the cause of all those duplicate ACKs. This explains the slower overall speed measured for download. Every time a packet is lost, we send ACKs for the last packet we DID receive while we wait for it's arrival, and the other system eventually re-sends it, and the stream can then continue again. But those little pauses extend the download a little at a time, until the measured bytes / time is dramatically slower because we stopped 80 times during the transfer. Now, here's the kicker... 80 packets lost out of 161,507 is only .05%, so Centurylink considers this "normal" packet loss and won't do anything about it. In fact, this is better than average for my service level. Don't believe me? Check out their publication: https://www.centurylink.com/aboutus/legal/internet-service-disclosure/full-version.html , it actually says:

"Packet loss (the percentage of packets that are sent by the source but not received by the destination) is also highly variable. The most common reason that a packet is not received is that it encountered congestion along the route. A small amount of packet loss is expected, and indeed some Internet protocols use the packet loss to understand Internet congestion and to adjust the sending rate accordingly. CenturyLink denotes a packet as lost if the latency exceeds 3 seconds or if the packet is never received. CenturyLink High-Speed Internet customers should generally expect to experience packet loss at the rate significantly below 1% or at levels unlikely to significantly affect customer experience."

Maybe in other areas CenturyLink has less packet loss, and others they may have more. The lesson I've learned is that you shouldn't just take your ISP's word for it that you're getting what you paid for. Go check it out. Are you getting the advertised speeds? What about reliability, are they losing your packets? I chose to stick with Cox because packet loss is a big deal to me. Not only is this going to affect download speeds, but this also affects reliability and latency of persistent connections. In other words, it plays a huge role in gaming experience. It isn't cool when you're about to win a battle and get lagged out or disconnected, only to log back in and find your character dead. This is what you can expect with "normal" packet loss. :-)