v1.5.21 Bug Reports and Comments

[JeffreyS]

I am also seeing these random power spikes on 2 switches. My sample data for this issue are 2 x WS-12-250-DC and 1 x WS-8-150-DC

The WS-8-150-DC is connected to a 24V batter/solar system. No spikes.

1 WS-12-250-DC is connected to a 48V battery/solar system. It has seen 5 spikes in the last 24 hours. 2 of which were in the middle of the night, no solar activity.

Other WS-12-250-DC is a 24V bater/solar system but is connected behind a 24V to 48V step up transfer as at the time of install the fuses available were too small to support the switch on the 24V rail so had to utilize the 48V rail. This switch should be seeing a constant 48V but yet sees spikes. It has seen 2 total in the last 24 hours.

My sample base is small but seems like it is related to high input voltage (48V).

@yahel, what input voltage are you running?

[yahel]

These voltage surges are happening on both 48v sites (often as high as 57v) and on 24v sites (often as high as 28v).
This is happening on hundreds of switches.
These are probably NOT real surges - they do not show on 1.5.14 at the same sites and switches.

[sirhc]

These voltage surges are happening on both 48v sites (often as high as 57v) and on 24v sites (often as high as 28v).
This is happening on hundreds of switches.
These are probably NOT real surges - they do not show on 1.5.14 at the same sites and switches.

I understand this but why are you avoiding answering the questions I ask you?

On or off grid
Solar or wind chargers if off grid (type of charge brand and model)
What type of power chargers if on grid
Number /type of batteries
Series or parallel
Grounding
The more detail of your setup the better.

Please explain your power setup in detail.

Regardless if this is or is not real spikes the more info we have the better.

You keep dancing around my questions and and keep repeating the same thing, this does not happen in v1.5.14, WE GET THAT, you have said this like 6 times.

But whos to say something was not fixed after v1.5.14 and POSSIBLY the spikes are real and started showing up after a fix?

I'm trying to work with you but if you are unwilling to answer my questions I am just going to put this on the back burner for now.

[sirhc]

I am also seeing these random power spikes on 2 switches. My sample data for this issue are 2 x WS-12-250-DC and 1 x WS-8-150-DC

The WS-8-150-DC is connected to a 24V batter/solar system. No spikes.

1 WS-12-250-DC is connected to a 48V battery/solar system. It has seen 5 spikes in the last 24 hours. 2 of which were in the middle of the night, no solar activity.

Other WS-12-250-DC is a 24V battery/solar system but is connected behind a 24V to 48V step up transfer as at the time of install the fuses available were too small to support the switch on the 24V rail so had to utilize the 48V rail. This switch should be seeing a constant 48V but yet sees spikes. It has seen 2 total in the last 24 hours.

My sample base is small but seems like it is related to high input voltage (48V).

@yahel, what input voltage are you running?

Thank you, finally some USEFUL info. Let me digest this. I may have follow up questions.

[mayheart]

I'm seeing the same graph spikes with a IDC and DC units with 1.5.21, doesn't seem to be affecting anything though.

IDC unit is connected to a Emerson -48V DC system. ~8-10ft cable run

2 DC units are connected to Meanwell +54v power supplies. (about 48 and 50v after voltage drop from the run)

[Flo]

The reason I am asking to try another brand or 2 of SFPs is we re-wrote how SFP modules are handled to fix previous issues.

Can you please provide me a v1.5.21-dev firmware including the original SFP module code for testing?
This would help me to figure out, if this issue is related to the latest SFP module code changes.

If you found a brand that caused issues then we could order same module and look why.

I can ship you some, in case the original SFP module code works on the latest firmware.

I meant RSTP not MSTP ...

Has there been a code change between v1.5.16 to v1.5.21 regarding _STP handling?

We are trying to help but am also asking you to try and help eliminate rabbit hole as we have limited time and NEED to get WS3 back on track or pack it up as industry WANTS 10G SFPs and 2.5G copper.

Yes, that would be perfect having new WS supporting 2.5G copper and 10G SFP+ with _STP support.

And WS3 line WILL have routing abilities, not right away but within months of pre-release including static, OSPF, and BGP but not world route capable. Also very impressive bandwidth limiting, up to 4096 rules per switch based on ingress /egress / IP / MAC / Port or range / timeand more.

Ohhh, that's awesome. Thank you for your outstanding work.

[yahel]

These voltage surges are happening on both 48v sites (often as high as 57v) and on 24v sites (often as high as 28v).
This is happening on hundreds of switches.
These are probably NOT real surges - they do not show on 1.5.14 at the same sites and switches.

I understand this but why are you avoiding answering the questions I ask you?


On or off grid

* About 20 out of 200 are off grid.

Solar or wind chargers if off grid (type of charge brand and model)
* No wind - only solar. Predominantly Victron chargers.

What type of power chargers if on grid
* So many different chargers at different site - mostly Victron.

Number /type of batteries
* some sites have 48v batteries, some are 24.
* ALL sites have LiFePO4 batteries

For example - most common large sites use:
https://www.amazon.com/dp/B0D8L6F3Z8?

Series or parallel -
* mostly single battery enclosure with BMS.
* We used to have dual AGM cells in series (or 4 in series) but we moved completely to LiFePO4.

Grounding
* We try to have excellent ground for all sites. All switches have a grounding cable to good local ground.
The more detail of your setup the better.

Please explain your power setup in detail.
* Let me know what else you might need to know?
* Again - we are talking hundreds of sites -- no surges with 1.5.14 -- all have surges with 1.5.16 (regardless of power setup).

Regardless if this is or is not real spikes the more info we have the better.

You keep dancing around my questions and and keep repeating the same thing, this does not happen in v1.5.14, WE GET THAT, you have said this like 6 times.
Not dancing - ask and you will be answered -- nothing to hide.

But whos to say something was not fixed after v1.5.14 and POSSIBLY the spikes are real and started showing up after a fix?
* That is a possibility - but at nearly 200 sites? With different power chargers, different batteries -- unlikely !!

I'm trying to work with you but if you are unwilling to answer my questions I am just going to put this on the back burner for now.

Have I answered all the questions? Why the attitude? Sorry I have not seen this note within minutes of you posting -- it's sometimes hard to follow these threads -- I'm sure harder for you.

Thanks!

Yahel.

[yahel]

Chris - here's a sample site...
Not the cleanest site - but were I was today to replace a bad fan in a Netonix WS-26-250-DC
Seen mounted on the wall, next to a CCR1009-PC
On the left two 4800Wh batteries in parallel ($2200 just for these batteries).
Two AC Victron chargers
A Victron "Cerbo" controller and a Victron shant (BlueTooth) to measure current.

[Flo]

just one more bug:
ssh to v1.5.21 and run:

Code: Select all

 {switch-name}# web_conf_check
sh: php-cli: not found

[Flo]

Hi sirhc,
I have noticed the following serious bug on v1.5.21:

SFP ports (at least on WS-12-250-AC) block multicast transmissions of PPPoE-Scan (PPPoE-Discovery, MAC proto 8863) packets, even though the MC option is enabled on both ends of the fiber link. Non multicast packets (like PPPoE, MAC proto 8864) do pass.

It does work correctly on RJ45 copper ports, so it seems to me that the issue is related to the latest SPF-interface updates.

Could you please try multiple SFP brands, espiciall if you have genuine Cisco modules and let us know. And if you find others work please send pic and info on module that fails. Also try disabling MSTP on that port, helps us out a lot.

Ok, did some further debugging on this issue. Hopefully the following results can help you to resolve it:

-> The issue only occurs, if the PPPoE server is connected on a SFP-port running latest v1.5.2x firmware series.
-> replacing SFP with other brand: same error
-> disabling STP on switch: same error
-> downgrade to v1.5.16: issue resolved

@sirhc: Do you have another idea, what may cause PPPoE discovery packets being filtered on SFP uplink ports running latest v1.5.2x firmware series?