[SOLVED] Lag spike issues with Engenius WAPs and PoE Switch

[Rogue Leader]

I'm losing my mind chasing this issue. First off the hardware:

Dlink DSR-250 Router https://us.dlink.com/en/products/dsr-250-8-port-gigabit-vpn-router
Engenius EWS7928Pv2 PoE+ Switch https://www.engeniustech.com/engenius-products/managed-poe-network-switch-ews7928p/
4 Engenius EWS357APv3 Access Points https://www.engeniustech.com/engeni...i-6-2x2-managed-indoor-wireless-access-point/

This is connected to and Altice One Router/Wifi/Gateway. The reason for this is that gateway is required to get 1gbps service from them, and it needs its own wifi signal for the cable boxes to work. The Wifi SSID of that gateway is unique, and 2.4 ghz is set to channel 11 while 5ghz is set to channel 157, both specifically so I can utilize channels outside of that range.

Another note, devices wired direct to the switch or Router have NO issues. I have isolated the issue specifically to Wireless devices connected to the APs.

The wireless APs have been set up based upon Engenius best practices: https://helpcenter.engeniustech.com...ng-wireless-roaming-issues-What-should-we-do-

I have 3 SSIDs on VLANs, properly configured. No IP conflict type issues. 1 of the 3 SSIDs is hidden as well. Also wise to note this issue has been happening from day 1, and through various configuration changes it seemed I had fixed it, except its randomness meant that was just a mirage.

So the problem. It seems my higher powered Wifi Devices (my work MacBook Pro, my wifes HP Envy X360 laptop, my sim racing desktop system) experience random lag spikes. By lag spike I mean in testing using Ping it will drop 10+ packets, or at a minimum those packets will return with a ping time of 4000ms, 3000ms, and so on down back to normal. Like it bottlenecks for a second or more. The HP Envy is most often affected and most strongly affected. We both use Citrix to remote into work servers, her laptop is downright frustrating to use.

Now heres the interesting part. I have a Microsoft Surface, I can ping google from that all day long and lose 1 maybe 2 packets. As far as I can tell the lower power devices (our phones, kids ipads, etc) are NOT experiencing these spikes. We can stream Netflix on our Roku all day long with 0 drops, but keep in mind stuff like that is buffered so that may prevent a major drop. And the more interesting part I was able to trigger such spikes a few times, but not consistently.

How I triggered a spike:

1. I had tried disconnecting one of the APs, I then plugged it back in, upon plugging it in it triggered a spike on the MacBook.
2. After it booted up I had my wife turn wifi on on her iPhone 11, this triggered a spike. I had her do it again a few minutes later and it did the same. Then I had her move with wifi off and stand next to another AP and turn wifi on and it did NOT trigger a spike. Note the second connection was at 5ghz instead of 2.4 for some reason.
3. Later on I had her walk from one part of the house to another with wifi on (5ghz) and that triggered a spike when her phone moved to another AP.
4. I managed to do the same thing with my OnePlus 7T Pro moving from one AP to another. But only once did it happen.
5. I triggered a spike running Speedtest from my OnePlus 7T Pro, once. Then never again.
6. Turning on the Racing System just now triggered a massive spike on the MacBook Pro (HP envy is still off)

The spikes are more common the more high power devices are together. Sit the MacBook and HP Envy next to eachother and The HP Envy spikes every couple minutes. However right now the HP envy is OFF completely and I've only experienced one spike in the past 10 minutes. That said yesterday I was able to get the HP Envy to sit with no spikes at one point for a long while (10+ min) after flushing the DNS and all those steps. Then I did it again and experienced 8 in a 10 minute period. All the test triggering I mentioned above was done with the HP Envy off. And again at the same time, my Microsoft Surface was pinging google non stop and experienced 0 issues.

I tried turning EVERYTHING 5ghz off except my MacBook and pinging with that, and it was pretty stable only happened once. Right now I just pinged google for 30 minutes form the Macbook and only had 1 spike.

I know the Altice provided gateway has a "SmartWifi" feature, I am wondering if that is doing any channel scanning which may cause such issues. It should not be doing that since I set it to specific channels but my control of that router is quite limited. I just figured out how to disable that feature so I'll keep testing it with it now off but it seems to still be happening. I really want to blame the HP Laptop here but the reality is with it off the spikes still happen, albeit quite a bit less (once or twice an hour). On my Race Simulator system I've been racing and not noticing any issues. Maybe just good error correction, but I'd say its not affected as strongly as the HP laptop either.

Literally losing my mind over this. Emailed Engenius Tech support but from what I've read they are awful about responding in a timely fashion.

Another note, I have until Jan 31 to return all this equipment, so if it comes down to it, I can dump it all and go with a different brand. Thanks.

EDIT: Per #6 I turned on the sim system and am running ping on it while using macBook. I walked into the room with the MacBook in hand and there was a small spike. I then sat down next to the Sim system and both experienced a massive massive spike and then were both consistently in and out. This subsided when I walked away, and now I'm just getting them every few minutes with Ping running on both machines.

 

[gggplaya]

Try turning off "FAST ROAMING" on your access points.

How big is you house? 4 wifi access points would be alot for a small house, and you may be roaming a lot between access points which would cause lag spikes. Too much overlap coverage may be an issue. Try turning off 1-2 access points.

 

[Rogue Leader]

Try turning off "FAST ROAMING" on your access points.

How big is you house? 4 wifi access points would be alot for a small house, and you may be roaming a lot between access points which would cause lag spikes. Too much overlap coverage may be an issue. Try turning off 1-2 access points.

Tried turning off Fast Roaming, didn't change anything.

House is 5500 Sqft, 3 WAPs on the ground floor and 1 on the second floor. But to your point, I'm going to try disabling the middle one on the first floor, see what kind of affect it has.

 

[Rogue Leader]

Try turning off "FAST ROAMING" on your access points.

How big is you house? 4 wifi access points would be alot for a small house, and you may be roaming a lot between access points which would cause lag spikes. Too much overlap coverage may be an issue. Try turning off 1-2 access points.

You, sir, are a legend. May riches fall from the heavens to you.

Disconnecting the middle WAP immediately solved the issue. I still do get some interference between the 3 remaining ones but I can tweak that by turning the radios down a bit on all 3 so they overlap less.

My gosh such a simple thing! I'm going to keep this WAP however and use it in the basement someday when we refinish it. right now if you go down there you pick up very little wifi signal at all other than the Altice router.

 

[Rogue Leader]

@gggplaya

So not totally solved yet. While removing the middle WAP made a MASSIVE difference, we are still in some areas of the house getting ping spikes using higher powered WiFi devices. I turned the radios down on everything quite a bit which helped, but it still happens sometimes. It even happened when my mother in law came over earlier and walked into the house with her iPhone in her pocket in one of the areas where there seems to be crossover. The spike was way more minor but still quite noticeable using Citrix.

Couple notes: the two remaining WAPs on the 1st floor (lets say the whole floor is about 2500sqft) are at opposite ends of the floor, the second floor WAP is dead in the middle. The two first floor WAPs are WALL mounted on outside walls of the housewhich is not the optimal position, so I'm wondering with the strongest signal going up down and out instead of left and right could that be where the fighting between WAPs is happening. Could mounting both 1st floor WAPs with a product like this:

https://www.tripplite.com/universal...s-access-point-right-angle-steel-white~ENBRKT

Help the situation?

Also tagging @bill001g as I know hes an expert in these things. Thanks so much gents.

 

[Rogue Leader]

https://www.tripplite.com/universal...s-access-point-right-angle-steel-white~ENBRKT

Help the situation?

I think these have resolved the issue for anyone looking. My devices seem to transfer between WAPs correctly new. And I seem to have no lag spikes. Tomorrow is a work day so that will tell if its solid with multiple higher powered wifi devices streaming (Citrix) at the same time.

 

[Rogue Leader]

I think these have resolved the issue for anyone looking. My devices seem to transfer between WAPs correctly new. And I seem to have no lag spikes. Tomorrow is a work day so that will tell if its solid with multiple higher powered wifi devices streaming (Citrix) at the same time.

Still seeing occasional dropouts. They seem to be driven by my wifes Laptop which is significantly more affected than my Macbook and any small network device.

Sitting next to her and running netspot on my phone I am seeing -60db on the closest WAP, and -81 on the 2nd floor WAP which is the next closest.

While I'm sure my fixes of removing the middle AP and putting the downstairs ones on their side have helped quite a bit, I don't know if her laptop is the final straw, or maybe something in my home is causing some sort of interference. There is a large refrigerator/freezer right in the middle of the first floor basically equidistant form all 3 APs.

Maybe there is some software that can show me interference that is NOT necessarily wifi APs?

 

[gggplaya]

At this point, since your AP's can do multiple SSID's. I would give each AP an SSID of it's own unique name. Like Bedroom AP, Living room AP, 2nd floor hallway etc.......

For devices that don't need roaming such as office computers and tv boxes, I would just connect to the nearest AP using the unique SSID. That eliminates any chance of roaming. If they are close enough, make them 5ghz only depending on coverage in your home to eliminate the chances of interference.

If you are still having problems. I would drop Engenius and swap to Ubiquiti or Ruckus.

 

[Rogue Leader]

At this point, since your AP's can do multiple SSID's. I would give each AP an SSID of it's own unique name. Like Bedroom AP, Living room AP, 2nd floor hallway etc.......

For devices that don't need roaming such as office computers and tv boxes, I would just connect to the nearest AP using the unique SSID. That eliminates any chance of roaming. If they are close enough, make them 5ghz only depending on coverage in your home to eliminate the chances of interference.

If you are still having problems. I would drop Engenius and swap to Ubiquiti or Ruckus.

One thing to note is I can see if I watch the active clients tab in the console when a device is roaming and being picked up by multiple APs. I tested this using my phone last night and a couple other devices. I can over time see the connection to 2 APs and the Mac Address listed twice and watch the transition. My phone kept a continuous connection and moved smoothly between APs.

I can kick clients from the APs from here as well so for example if I kick my daughter's nanny cam I can watch it work through the 3 APs until it finds the one with the best connectivity and then it stays there.

Monitoring active clients while the spikes happen to her, I am not seeing her laptop getting picked up by the other APs. So it doesn't seem that shes roaming or having a roaming issue.

Just now she experienced multiple lag spikes, and I am sitting next to her with my Macbook. Before the changes made this weekend if she saw a spike I would see one on my system every time too. But shes now seen multiple spikes with me sitting here and none have affected my laptop. We are both about 10 feet from the refrigerator (which I am still suspicious of lol).

Your idea is sound, but I feel like being that I can prettymuch see roaming isn't the issue, its gotta be something else.

EDIT: Side note, I just looked and I can do basically the same thing i have done using Ubiquiti parts for the same price. I am quite tempted as I am within the return window still.

 

[gggplaya]

Try moving the wife or the fridge for a day, to see if it still happens.

I have not experienced any of those with my Ubiquiti AP's, and I do actually have alot of overlap between my access points. The roaming works excellently as well when I move between floors.

 

[Rogue Leader]

Try moving the wife or the fridge for a day, to see if it still happens.

I have not experienced any of those with my Ubiquiti AP's, and I do actually have alot of overlap between my access points. The roaming works excellently as well when I move between floors.

Fridge doesn't move. Moved the wife to 5 feet away from the AP. Still has the same problems. Problems were worse when we were sitting next to eachother. But they still happened to her with us apart often enough.

My roaming works great actually now that I've made all these changes. The freezes though, its gotta be hardware somehow. Maybe something with the switch, I just can't otherwise explain the issues, it just doesn't make sense.

I'm about 90% made up to return it all and replace with 3 UniFi NanoHD and a UniFi Switch 24 PoE. Will be basically a net 0 cost, and a better UI. I used their design tool and with the same locations I have WAY less overlap than even the examples they show.

 

[Rogue Leader]

After talking with some network admin friends, I'm convinced its a bug either in the firmware of the Controller or the APs themselves.

Suffice to say I'm done with this game, its all going back in the box and back where it came from.

I ordered the aforementioned Ubiquiti hardware and it should be here by the end of the week.

Thanks again @gggplaya for all your help.

 

[Rogue Leader]

If anyone wants to know the final resolution, the replacement of the network hardware has completely cured the issue. The bug must have been in the firmware of the Wireless Controller portion of the switch as it affected all APs.

I returned everything and replaced it with the Ubiquiti Dream Machine Pro Router, Ubiquiti 24 port PoE switch and 3 Ubiquiti NanoHD APs.