High Latency When Only One Person Uses Internet

[Norott]

Our internet has been very unstable for about the past half year from what I remember. I rarely played online games with college being college, so with summer coming up it would be great if I could finally lay this problem to rest.

Specs:
-Netgear WNDR4300 Router with OpenWRT firmware
-Seven devices. 3 wireless tablets, 1 wireless wii, and 3 wired PCs Only 5 are ever used at once.
-Connection type is Broadband
-I am using windows 8.1, while the other 2 PCs are Windows 7
- 15mbs down 3 mbs up and i get roughly 20 ping with no one else on via speedtest.
- Using pingtest.net Packet loss comes out to 0%, ping roughly 60 to eastcoast, 80 to westcoast, and jitter ranges from 10-30ms

The problem:
I can be playing a game on steam with nice stable ping of 20-80ms but if I just try to use skype it will skyrocket and jitter. This also happens when another family member uses Netflix or Twitch. The same thing happens when my brother is playing games on his computer, and it just seems to be ping that gets decimated. The person on twitch/netflix seems unaffected through all of this, only games seem to get hit hard. With 2 people on netflix, and 2 people streaming on high/source on twitch, the streams don't lag. Our raw download seems good. So what is causing my ping to get decimated as soon as someone else gets on with 15 down and 3 up?

There was also a couple of cases of rubberbanding even with decent ping. Definently on Steam servers

What I tried so far:
The main problem I looked into over the past week was buffer bloating. I ran the ICSI Netalyzer and the results showed 1200 uplink ms and 3700 downlink. which seems quite high. After researching buffer bloating i found you can use openWRT's QoS to help mitigate the problem, but it didn't seem to do anything once I changed with some of the parameters. I decided to ask here before I really do screw something up since the other 'fixes' were confusing for me.

Ultimately,can openWRT really help with buffer bloat? Is bufferbloat part of my problem? Or am i just barking up the wrong tree? Thanks in advance for the help!

 

[richb-hanover]

...Ultimately,can openWRT really help with buffer bloat? Is bufferbloat part of my problem? Or am i just barking up the wrong tree? Thanks in advance for the help!

YES! The Barrier Breaker (14.07) release of OpenWrt can solve Bufferbloat problems.

You can confirm whether your router is bufferbloated or not by going to the DSLReports Speed test at http://dslreports.com/speedtest Run it, and if the Bufferbloat increases while downloading or uploading, then your router's not serving you well.

The good news is that the SQM/fq_codel package in OpenWrt will completely address this. Read the Bufferbloat page on the OpenWrt wiki at: http://wiki.openwrt.org/doc/howto/sqm

In a few clicks, your network connection will be much more responsive, even when uploading and downloading

 

[Norott]

Thank you! I followed the instructions and it seemed to help so far. The DSL report did test positive for bad bufferbloating and I will do some more tests over the weekend.

I am going to reason this helps bufferbloating by and making a form of overhead by limiting the down and uploads by 95%. Now, what if my internet's speeds plummet. I can get 15 mbs down and 3 up most of the time, but on occasion it does dip to 10mbs or so. I did check to see that no one was on, I am certain the dips are caused by something on the ISPs end out of my control (rural areas).

If I set the limit to around 14 mbs down, if the internet dipped so it could only provide 10-13 mbs; would bufferbloating occur again?

 

[richb-hanover]

No, bufferbloating wouldn't occur if the link speed drops. Here's why:

fq_codel knows how long it should take to send packets (because you entered the link speeds). It looks at the time the packet stays in the buffer, and if a flow's packets are taking too long, fq_codel begins to slow that particular flow (connection) either by dropping some of that flows packets, or marking with ECN to tell the sender to slow down.

When your link is fast (say 3 mbps up) and fq_codel sees packets that have been there too long, it starts dropping.

fq_codel doesn't care if your link speed drops, say to 2mbps, or if another device also creates traffic. fq_codel still measures the *time* that a flow's packets have hung around in the queue: if they've been there too long, they'll get dropped/marked as usual. Your total data rate might go down, but there won't be a lot of packets in the queue, and therefore no bloat.

One other thing: You definitely want to set the link speeds (both ways) as high as you can without inducing bloat. Repeat the pattern: increase the link speeds in the web GUI, then run the DSLReports Speed Test until the latency gets high, then back off a bit to set the router to the highest sustainable speed.

 

[Kewlx25]

Bufferbloat will start to happen again. codel does not know how fast your link is, you have to tell it how fast it should be.

If your upload drops to 2Mb/s and you're uploading 3Mb/s, bad things will happen. Either you take this on the chin or you set your upload to 95% of 2Mb and not 95% of 3Mb.

Codel can stop packets from getting to old in your routers buffer, but not in your modem's buffer. In order for this to work, your router must be limited to be slower than your modem.

If everything goes correctly, your buffer bloat should look more like this

richb-hanover: A lot of people get the dynamic bandwidth issue wrong. A lot of the examples the codel designers have shown where with codel built directly into the modems, so you didn't need it in the router. The modem knows exactly what the current available bandwidth is, so fluctuating bandwidth is not an issue. Your router on the other hand is using Ethernet, which only has one link speed, so this information is not known to the router.

Late Edit: I see this post was mentioned in bufferbloat.net and one thing discussed that stood out was a potential way to accommodate fluctuating bandwidth is if your router/firewall supports pinging an external target, preferably a low latency target in your ISP, and adjusting bandwidth based on detected latency changes.

Essentially reactively adjusting for bufferbloat. I am not familiar with any setups that do this, but there must be some out there.

 

[richb-hanover]

Bufferbloat will start to happen again. codel does not know how fast your link is, you have to tell it how fast it should be.

If your upload drops to 2Mb/s and you're uploading 3Mb/s, bad things will happen. Either you take this on the chin or you set your upload to 95% of 2Mb and not 95% of 3Mb.

Codel can stop packets from getting to old in your routers buffer, but not in your modem's buffer. In order for this to work, your router must be limited to be slower than your modem.

If everything goes correctly, your buffer bloat should look more like this

richb-hanover: A lot of people get the dynamic bandwidth issue wrong. A lot of the examples the codel designers have shown where with codel built directly into the modems, so you didn't need it in the router. The modem knows exactly what the current available bandwidth is, so fluctuating bandwidth is not an issue. Your router on the other hand is using Ethernet, which only has one link speed, so this information is not known to the router.

I have thought more about this, and you're correct that in this case, the modem's buffers will get bloated (and affect your performance.) However, that's not something that you can really control. You'll still be better off having SQM/fq_codel in place than not.

Your point about the modem knowing the actual ink rate is further evidence that *every* piece of networking gear (home router, DSL modem or DSLAM, Cable modem or head end, and even commercial routers and switches) all need to have some form of SQM to prevent bufferbloat.

Some vendors get it, but there's a long way to go. DOCSIS 3.1 gets praise for specifying PIE as a countermeasure for cable modems, but very low marks for its incredibly slow rate of roll-out.

I remain hopeful that we can pressure our vendors into solving this problem, now that we know what's causing it.

PS That's pretty impressive report from DSLReports. Mind telling what kind of router/internet connection you have?

 

[Kewlx25]

PFSense with a dedicated connection. I am unsure what kind of AQM my ISP uses, but it seems to work just fine. Even without my router, I get about 10ms of buffer bloat, with my router, well, that picture tells the story.

 

[Norott]

Sorry for the late reply was expecting gmail notifications and never got any. I might have figured out both my problems, with your guys help of course.

As for the internet dropping speeds during the day, it might due to interference between the tower and my satellite by a silly tree. I am within walking distance of it, but there is a tree in the way. I called my ISP and ran some tests with the WAN port directly in my computer and speeds were still erratic. They are coming out and putting up a stronger satellite to help that problem. They put up the satellite during winter when no leaves were on the tree and the speeds were fine during winter. So they set up a weaker satellite.

And on top of that I think bufferbloating was also happening. My ISP is coming out this thursday and hopefully help with the signal strength first.

All in all, my bufferbloating problem did go away for the most part when i did have the 15 down. So hopefully I if i get a more stable connection then I can more accurately solve the bufferbloat problem. Thanks for the help guys!