Question The impossible internet problem of 3 years

[vguy95]

Hi, I'm going to do my best to explain what I know about this situation I'm in. I would really appreciate if the Networking genius' could help me figure this problem out, since we've had this similar problem for 3 years. Its a direct ethernet connection, not WiFi. Its 1 GB down and 100 upload from Shaw here in Canada. I have ran tests on other computers in the household, with different ethernet cables and ports and created the same situations. The modem/router is the same device.

It all began 3 years ago, out of no where between like 2 am central to 11 am, the internet would spike and lag on things like live streams. I complained many times, and eventually some guy on the forums said they found the problem and patched it, seemed to work. About 3 months later, the problem was less specific to time, but my upload would spike downwards to 49 Kbps or so. Because I'm a live streamer on Twitch.TV its important to have a good connection, so when I was running Streamlabs OBS, I noticed I was dropping 3-5% of my total frames, which is very bad for streaming. In order to see what exactly was going on, I went to inspector twitch TV and graphed my upload. It showed what I suspected, the upload kept spiking downwards to 49 Kbps, sometimes 1 Mbps, sometimes 2 Mbps, sometimes 3 Mbps -- and you need about 6 Mbps upload to maintain a 1080p stream. I also tested other twitch servers, the same thing.

When I contacted Shaw they did the typical hey, I did a speed test it looks fine. They hooked up their equipment to my cable modem, said hey it looks fine. In the past, they had sent a total of 10+ techs out, and locally they could never find an issue. About a year later, someone working internally at Shaw almost instantly found a problem in the node. He called me personally, and let me know that he found the problem. The problem was like 90% better. He then found some other problems, but it was still doing this weird spiking stuff once in awhile. It was never completely fixed, but I was somewhat satisfied and just accepted it.

Some months later pass, I end up with the exact same problem. The upload is spiking downwards again. Because Shaw didn't believe my Streamlabs OBS and inspector Twitch, I decided to watch my Streamlabs OBS live information, and when it would begin lagging I would immediately go do a test on testmy.net, with a dummy file upload. This way I could try to rule out whether its a specific server, or if its just like this across the entire internet for me. Well, when I did that and plotted the graph there too, it showed my average upload was 8.8 Mbps, and the graph showed spikes downwards to 1 Mbps (as my other applications were reading).

I showed Shaw all of this. They keep saying it looks fine, no matter what I do. So, my next step was I was going to try to find even more evidence to show a problem exists. So, I downloaded pingplotter and did some tests. It showed at 10.0.0.1 (my local modem) has a constant 50% packet loss. When I do pings to google, theres many hops that have a constant 50-70% packet loss too. I notified Shaw of this, and they said they dont trust 3rd party software for their diagnostics.

One thing I'm questioning, is that theres a cable line going directly to my house, which is close to my modem, that looks frayed. The tech was up there just the other day, I'm not sure how he didnt notice that, or maybe its not a big deal and I'm wrong about that.

Here are some of the images I have. (? when I try to upload it says contact admin something went wrong)...

https://imgur.com/oOXXyXF

View: https://imgur.com/oOXXyXF

https://imgur.com/jmQ7Nxf

View: https://imgur.com/jmQ7Nxf

https://imgur.com/SOZtis7

View: https://imgur.com/SOZtis7

https://imgur.com/nRIPRUb

View: https://imgur.com/nRIPRUb

https://imgur.com/rR5pcrG

View: https://imgur.com/rR5pcrG

 

[Ralston18]

May well be hidden but I do not know and do not have access to a "Shaw Modem". The "Shaw Modem" could be some other make and model modem but just branded Shaw. Still the ISPs often have their version of firmware installed to suite their own purposes and objectives.

Perhaps someone else with a Shaw Modem can provide the necessary menu path to the DOCSIS tables.

Pingplotter may be showing problems. However, I am becoming much less convinced that that is indeed the case.

Three reasons:

1) The pingplotter results can be and are questionable. Plus subject to errors in both the testing process and the interpretation of those results. All of which was well noted in earlier posts.

2) The current Upstream speeds (9.8 Mbps and 8.3 Mbps) are actually quite in line with and even exceed the speeds presented via the links I posted in Post #72.
Plus I have reached the likely late conclusion that the comparison speeds (grey bars) being made may include commercial customers. Not just residential customers.

3) There is no other data comparative or otherwise indicating a problem. With the possible exception of the Channel 2 outlier. Which may be moot or a simple "red herring". DOCSIS tables could reveal some potential problem - yet, if so, then the Upload Stream speeds would likely be much lower than currently being "measured".

Barring other posted comments and suggestions, I am now at an impasse here.

 

[vguy95]

In reply to your 2nd comment

That is an AVERAGE. If it spikes up to 30 Mbps, and then down to nearly 0, then you still would average 15. The problem is the spikes downwards to 0 Mbps. The overall average IS BAD because of the spikes themselves though. You have to look at the graphs I post, where it shows the upload is going 20-30 Mbps in some situations then suddenly goes to 0. I have had other people run upload tests, their graphs never look like this.

 

[bill001g]

This is a huge thread so I will only respond to your last point.

All network numbers are averages. Things like ethernet ether send the data full rate, say 1gbit or at zero. If you did not average them the number would be pretty worthless. Problem is if you choose a pick a time to average them over that is too long then it could hide problems.
This is made even more complex when you look at say a application level. Something like netflix or youtube will send pretty large buffers of data at your maximum internet rate and then send nothing. This helps hide problems with data loss and retransmission while still keeping the average rate high enough to prevent stalls on the user viewing. There really is no univerisal method of picking how to average data it varies a lot. This is also why you can not directly compare the results from all the various testing tools you find on the internet.

Your problem is you have to really understand this in detail. From the little I know about streaming even though you may create the data as a stream it is not really send to the server in that format. I mean things like my security camera send the data out as UDP. Although you can transmit this over the internet with effort it generally is not done that way. Most applications use TCP and many thing additionally send it over HTTPS data format both to encrypt it and to allow it to pass through firewalls etc.

Your very first step is to make sure that the data leaving your machine is actually being sent with no delays. You will likely need to capture the data so you can run tools that can in detail analyze the delays between packets you can then adjust the method of computing the average any way you choose.

This though is not always as simple as say when I capture my security camera data. The security cameras in theory produce a UDP packet at a fixed rate and I can see this...not that you can do much when the CPU in the camera overloads and does not send as many frames. In your cases it is much more complex it since it is not a simple UDP data stream. You are going to have to find the documents that show what the data stream is suppose to look like at the very detailed packet level.

The key reason you want to spend a lot of effort ensuring your machine is really transmitting correctly is that really is the only thing you can fix. You actually need to hope it is some setting on the machine causing it.

Look at it this way. Say I send a lot of packages with the postal service. They have so called delivery times some of which they guarantee and others they do not. If they lose or damage packages or they get delivered late the best I can do is get my money back. I can complain but they will do nothing. Maybe a large customer like amazon could put more pressure on them but even amazon can't actually stop using the postal service to delivery their packages no matter how bad the service might be. This is similar to your problem your only real option is to cancel your service and get another ISP.

 

[vguy95]

This is a huge thread so I will only respond to your last point.

All network numbers are averages. Things like ethernet ether send the data full rate, say 1gbit or at zero. If you did not average them the number would be pretty worthless. Problem is if you choose a pick a time to average them over that is too long then it could hide problems.
This is made even more complex when you look at say a application level. Something like netflix or youtube will send pretty large buffers of data at your maximum internet rate and then send nothing. This helps hide problems with data loss and retransmission while still keeping the average rate high enough to prevent stalls on the user viewing. There really is no univerisal method of picking how to average data it varies a lot. This is also why you can not directly compare the results from all the various testing tools you find on the internet.

Your problem is you have to really understand this in detail. From the little I know about streaming even though you may create the data as a stream it is not really send to the server in that format. I mean things like my security camera send the data out as UDP. Although you can transmit this over the internet with effort it generally is not done that way. Most applications use TCP and many thing additionally send it over HTTPS data format both to encrypt it and to allow it to pass through firewalls etc.

Your very first step is to make sure that the data leaving your machine is actually being sent with no delays. You will likely need to capture the data so you can run tools that can in detail analyze the delays between packets you can then adjust the method of computing the average any way you choose.

This though is not always as simple as say when I capture my security camera data. The security cameras in theory produce a UDP packet at a fixed rate and I can see this...not that you can do much when the CPU in the camera overloads and does not send as many frames. In your cases it is much more complex it since it is not a simple UDP data stream. You are going to have to find the documents that show what the data stream is suppose to look like at the very detailed packet level.

The key reason you want to spend a lot of effort ensuring your machine is really transmitting correctly is that really is the only thing you can fix. You actually need to hope it is some setting on the machine causing it.

Look at it this way. Say I send a lot of packages with the postal service. They have so called delivery times some of which they guarantee and others they do not. If they lose or damage packages or they get delivered late the best I can do is get my money back. I can complain but they will do nothing. Maybe a large customer like amazon could put more pressure on them but even amazon can't actually stop using the postal service to delivery their packages no matter how bad the service might be. This is similar to your problem your only real option is to cancel your service and get another ISP.

I understand how averages work and are calculated, the point I'm trying to make is 6 Mbps is around the minimum you need to have a live stream going at 1080p. So lets say it drops to 5 Mbps, you will have many dropped frames. This doesn't look good to the audience. Nowadays, it is not difficult for internet to maintain those speeds, unless theres a serious problem. On the same provider, I had 15 upload on my plan and I never had an issue. Now that they upped the plans, mine is currently 100 upload, there has been problems galore. The graphs I produce show the upload is constantly spiking downwards to less than 1 Mbps, which is absolutely atrocious. There is a discord with hundreds of streamers in my city, who use my provider, who dont drop a single frame their entire stream.

And even compared to 2 months ago, my upload is going 1/3rd the speed it was going, and even then I was having problems. Its actually getting worse and worse.

I'm confident it isn't my computer/applications causing this lag. I had this exact same problem before, and I had to convince Shaw there was a problem, eventually after screaming bloody murder for an entire year, they took it seriously and started fixing the noise in the node which was causing me issues. The person called me who fixed it, and told me to check my graphs again, they were almost completely cleared up. One of them referred to my node as a "recurring offender". I also created the same problem on a completely different computer.

What I want to know, is how can I prove that its in the node? That DOCSIS engineer implied that the pingplotter was steering him in the direction that he believes its in the node. If I can prove its in the node somehow, then I can at least try to take that to Shaw and show them. Have you heard of Smokeping?

 

[bill001g]

Unless you control the far end ping is not a reliable testing tool for many reasons. Ping can be used as a denial of service attack so many router and servers limit the amount of traffic they will respond to. So loss or delay in ping can sometime be the firewall software trying to protect the device.

Docsis problems do not generally cause delays. Docsis unlike say wifi does no form of error retranmission. Docsis does limited amounts of single bit data correction but it will just drop most damaged data. This will be seen as packet loss not a delay or lower bandwidth rate.

This is the problem with the tool you are using. Does it see data loss and just calls that lower bandwidth or is the data transmission rate slower.

Note the ISP equipment will clearly show data loss on the uplink just like your modem should clearly show the number of packets that have correctable and uncorrectable errors on the download. Now you would hope the ISP are not idiots but you never know.
If the ISP sees no packet loss ie uncorrectable errors at the node for your connection then the docsis is making no difference.

In any case since you can't get into the node itself to see things any anything you can test will be past that point you can never PROVE it is the ISP equipment. They can always blame the other ISP or some other equipment that is not theirs. In addition you are looking for the lazy way out. You are either going to have to find a service that does more detailed testing or you are going to have to say rent a virtual server so you can actually see the traffic being received. It will be much better than trying to say test against a router IP that has denial of service software running on it.

 

[vguy95]

Unless you control the far end ping is not a reliable testing tool for many reasons. Ping can be used as a denial of service attack so many router and servers limit the amount of traffic they will respond to. So loss or delay in ping can sometime be the firewall software trying to protect the device.

Docsis problems do not generally cause delays. Docsis unlike say wifi does no form of error retranmission. Docsis does limited amounts of single bit data correction but it will just drop most damaged data. This will be seen as packet loss not a delay or lower bandwidth rate.

This is the problem with the tool you are using. Does it see data loss and just calls that lower bandwidth or is the data transmission rate slower.

Note the ISP equipment will clearly show data loss on the uplink just like your modem should clearly show the number of packets that have correctable and uncorrectable errors on the download. Now you would hope the ISP are not idiots but you never know.
If the ISP sees no packet loss ie uncorrectable errors at the node for your connection then the docsis is making no difference.

In any case since you can't get into the node itself to see things any anything you can test will be past that point you can never PROVE it is the ISP equipment. They can always blame the other ISP or some other equipment that is not theirs. In addition you are looking for the lazy way out. You are either going to have to find a service that does more detailed testing or you are going to have to say rent a virtual server so you can actually see the traffic being received. It will be much better than trying to say test against a router IP that has denial of service software running on it.

So what would you do in my situation, knowing that in the past, this same problem (or an identical acting problem) was found to be in the node, even after they sent 10 techs in the past saying there was nothing wrong with the internet. This situation seems the same to me. Every real world test, shows garbage speeds, shows spikes in the upload downwards to 0 Mbps, over 300% slower than it was just 2 months ago... and they have a monopoly on the services here.

Just RECENTLY, there was a company called Vmedia that rents Shaw infrastructure that can provide the same internet to me. At this point, aside from convincing them theres a problem, the only thing I could do is switch to Vmedia, which uses Shaw infrastructure (so they will use the exact same equipment I believe, coming to the house) and then if I have the same problem, then I have to try to convince THAT company theres an issue and see if THEY take it seriously.

If I could personally put money on it, I'd say its in the node (again, like it was last time).

 

[bill001g]

You never know maybe you get lucky with the other ISP and it works. Maybe you can convince them that there is a problem and then you have a bigger company telling shaw there is a problem with their equipment.
The problem could be I guess that shaw knows exactly what the problem is and it is too costly for them to fix it so they would rather you just cancel your service than they fix it.

If I was in your situation I would be trying a hosting service so I could actually see the traffic. There are many very inexpensive services that let you rent servers. Even amazon is not too expensive, you will have to shop around if your bandwidth usage is a real lot per month.

What a lot of people do with these is use them as a private vpn services. Just using any kind of vpn might hide your problem because the data format is different as it passes through your ISP. You have your choice of few vpn to try, you can use things like IPSEC and/or openvpn on UDP. UDP data traffic is actually much better for data streaming because it is pretty much send and forget. Unlike say a file upload you do not really care to fix a damaged video frame because of the time it takes. If your ISP has some strange issue with TCP it would hide that.

In addition you now have much more ability to see the actual traffic rates. Many vpn servers have very detailed data on transfer rates and some even show the latency between packets.

Since you are not doing this for security reasons you likely can use some very simple tunneling protocols like GRE or run IPSEC without the encryption. It allows you to capture and actually look at both the "vpn" packets as well as what is inside. You could then compare the delays between the vpn data stream and the application data inside the vpn session.

 

[vguy95]

I have an update. My friend has the same Shaw plan as me (1 GB Down 100 Up) but hes in a different province, has the exact same router as me XB7.

https://imgur.com/2IxSHbe

View: https://imgur.com/2IxSHbe

https://imgur.com/SS85u25

View: https://imgur.com/SS85u25


Why would it be so different than mine? I noticed there isn't really a mixing of the QAM modulations like mine?

 

[Ralston18]

What are his modem's Downstream values?

His modem/router may being using a different firmware version, could be configured differently, and all components are working correctly.

Your modem/router may have a different firmware version (or may have the same firmware version but the firmware is corrupted/buggy), and/or incorrectly configured. Or some component is not working correctly.

Different Upstream power levels as well.

Lots of details involved - for Motorola:

https://motorolacable.com/whitepape... some networks, the power,for 2 to 4 channels.

More generic:

https://www.speedguide.net/faq/what-cable-modem-signal-levels-are-considered-good-78

Even though both modems may be XB7 the internal components could be different with only the case and branding being the same.