Flow Control issue?

User avatar
cwachs
Experienced Member
 
Posts: 115
Joined: Fri Nov 06, 2015 9:04 pm
Location: Colorado
Has thanked: 2 times
Been thanked: 10 times

Flow Control issue?

Tue Jul 25, 2017 10:54 pm

While trying to trouble shoot a different problem, I took a look at my port details on a couple switches, specifically those feeding ePMP APs (mix of 2000 and 1000's). I am seeing what looks like very excessive pause frames. This is fairly consistent across all of our switches on ports that feed our APs. None of our backhaul links have flow control on (either don't support it or they are AirFibers and we turn of FC).

Both of these are AP ports on different switches with many weeks of data collected:

Screen Shot 2017-07-25 at 8.29.47 PM.png


Screen Shot 2017-07-25 at 8.49.32 PM.png

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Flow Control issue?

Wed Jul 26, 2017 3:38 pm

If you run a large flat network turn Flow Control OFF

If you have a router at every tower and you know what Flow control is doing and how it works and you make sure backhaul radios and APs (local tower radios) never share a common port you can use Flow Control and it is a benefit.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
cwachs
Experienced Member
 
Posts: 115
Joined: Fri Nov 06, 2015 9:04 pm
Location: Colorado
Has thanked: 2 times
Been thanked: 10 times

Re: Flow Control issue?

Wed Jul 26, 2017 4:27 pm

This portion of the network is flat but medium sized. However, as I did deeper into this, the switch ports with Cambium ePMP APs have huge pause numbers on the RX side - across the board. The ports with Mimosa A4 APs have very small pause numbers - some of them at 0 after weeks of use.

If I turn off flow control at the switch port on a ePMP AP and reset the counters for that port - the RX Pause Frames rocket right back up.

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Flow Control issue?

Thu Jul 27, 2017 11:46 am

What firmware version are you running? (make sure you're using v1.4.8rc7 as we can not look at or do anything other than with the current version in development)

Your saying if you turn OFF FC for the port facing the epmp that port is still seeing "Rx" Pause Frames?
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Flow Control issue?

Thu Jul 27, 2017 11:48 am

Please post up your Main Status Tab and Device/Status Tab
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
cwachs
Experienced Member
 
Posts: 115
Joined: Fri Nov 06, 2015 9:04 pm
Location: Colorado
Has thanked: 2 times
Been thanked: 10 times

Re: Flow Control issue?

Thu Jul 27, 2017 12:34 pm

We are running 1.4.7 on production switches - we stay away from release candidates unless they fix something specific for us.

Here are screen shots from one of our switches. All of the switches connected to Cambium ePMP APs are acting the same (mix of 1000 and 2000 APs).

You can see Port 6 does not have flow control on. However, in the port detail page, you see tons of Rx Pause Frames. I reset the counters on this port last night. The other odd thing is the Rx Pause Frame # matches almost exactly the Rx Multicast number. When I reset the counters, I can watch both of those fields count up at exactly the same rate. And I see this on ALL of our ePMP devices. It does not matter if flow control is on or off for those ports. I do not see this behavior on the switch ports feeding Mimosa APs.

Screen Shot 2017-07-27 at 10.27.59 AM.png


Screen Shot 2017-07-27 at 10.27.49 AM.png


Screen Shot 2017-07-27 at 10.28.23 AM.png

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Flow Control issue?

Thu Jul 27, 2017 12:38 pm

viewtopic.php?f=17&t=2931&p=19985&hilit=+release+bug+rc#p19985

But beyond that not running v1.4.8rc7 on a DC switch is inviting a thermal shut down as it fixed a bad bug with the DC switches.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
cwachs
Experienced Member
 
Posts: 115
Joined: Fri Nov 06, 2015 9:04 pm
Location: Colorado
Has thanked: 2 times
Been thanked: 10 times

Re: Flow Control issue?

Thu Jul 27, 2017 1:06 pm

OK, took a different switch to 1.4.8rc7 (a WS-6-MINI). It has a ePMP 1000 AP on port 4. Reset all counters with flow control off (has been off). Here are the results. Same issue:

Screen Shot 2017-07-27 at 11.03.42 AM.png


Screen Shot 2017-07-27 at 11.03.34 AM.png


Screen Shot 2017-07-27 at 11.03.48 AM.png

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Flow Control issue?

Thu Jul 27, 2017 2:29 pm

This could be that the device (radio) is not properly acknowledging that FC is turned OFF and is sending Pause Frames regardless.

If this occurs the switch would count them as we are seeing but ignore them.

Try turning FC off on the radio and see what happens.

I can confirm that FC is working as it should with other radios such as UBNT, when FC is turned off on the port the UBNT radio is not sending Pause Frames. (Even though UBNT still has a broken Flow Control on airMAX AC and AF radios where it sends a gazillion Pause Frames/Storm under some conditions).

My guess is your epmp radio is ignoring that FC is not negotiated and is sending them regardless and our switch is counting them but ignoring them.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
cwachs
Experienced Member
 
Posts: 115
Joined: Fri Nov 06, 2015 9:04 pm
Location: Colorado
Has thanked: 2 times
Been thanked: 10 times

Re: Flow Control issue?

Thu Jul 27, 2017 2:51 pm

Wish I could turn off FC at the AP. Cambium ePMP radios do not give you a setting to do that.

What interests me as well is the pause frames = multicast frames on the Rx side. And it has nothing to do with a busy network. With little to no traffic passing across this network segment, the AP will send pause frames with every multicast frame.

I think this discussion now needs to be finished on the ePMP forums since it seems to be isolated to ePMP hardware. My UBNT and Mimosa hardware is functioning as I would expect it to with Netonix.

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 30 guests