Switch rebooting periodically......

DOWNLOAD THE LATEST FIRMWARE HERE
User avatar
jjonsson
Associate
Associate
 
Posts: 337
Joined: Wed Nov 05, 2014 12:30 pm
Location: Denmark
Has thanked: 37 times
Been thanked: 65 times

Switch rebooting periodically......

Sun Dec 03, 2017 8:41 am

I have Netonix switches at all my sites/towers. I'm very satisfied...

Unfortunately I have 2 sites where the switches reboot periodically. We have grounded everything and we have used Ohmmeter to measure that grounding is working correct.
What we have found is that at both sites, owners has solar cell system. Could it be something with the inverters for the solar system causing the switches to reboot randomly ?
If yes, how can we measure if that is the case, and how can we prevent it from happen ?
Of course this is only a wild guess, everything else has been tried without luck. And again grounding is checked by independent electricians....

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Switch rebooting periodically......

Sun Dec 03, 2017 9:50 am

Model?
MAC?
Firmware version?

The easiest way to determine if there is an issue with the switch itself is to swap it out with another spare unit and see if the problem continues or stops.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Switch rebooting periodically......

Sun Dec 03, 2017 10:04 am

There are some known issues that occurred with a few model which require RMA to fix. Not saying this is your issue but explaining why we require more information "than my switch reboots". - LOL

WS-6-MINI, WS-8-150-AC, and to a limited extent WS-8-150-DC had a production run awhile back (about a year ago +/-) where several hundred units had incorrect 3.3V CAPs which caused reboots. The reboot issue was more prone to occur the more ports that had links (USING POE OR NOT HAD NO EFFECT ON ISSUE SIMPLY THE LINK ITSELF) . Some units worked fine with all 100M links then acted up when radios were upgraded to 1G links. This issues was talked about a long time ago and we offered to cover this issue for the life of the unit so if you discover today or next year we will fix it free.
viewtopic.php?f=17&t=2975&p=20206&hilit=CAP#p20206
viewtopic.php?f=17&t=2796&p=19345&hilit=CAP#p19345


WS-12-250-AC units had one production batch about a year ago +/- where the power supply manufacturer had a bad CAP at C51 position on control board which would cause the switch to reboot. Usually occurred if you had a light POE load such as 30-60 watts or less. It started occurring after being powered up anywhere from 1 hour to 12+ hours. Usually this issues starts rebooting and gets worse until the switch locks up but f it cools down (the power supply) it works again as normal for 1 to 12+ hours then starts again. Once again we told people what to look for and said this issue is covered for life of unit.

viewtopic.php?f=17&t=2737&p=19310&hilit=C51#p19310
http://forum.netonix.com/viewtopic.php? ... C51#p18205
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
jjonsson
Associate
Associate
 
Posts: 337
Joined: Wed Nov 05, 2014 12:30 pm
Location: Denmark
Has thanked: 37 times
Been thanked: 65 times

Re: Switch rebooting periodically......

Sun Dec 03, 2017 5:39 pm

sirhc wrote:Model?
MAC?
Firmware version?

The easiest way to determine if there is an issue with the switch itself is to swap it out with another spare unit and see if the problem continues or stops.


Site 1 has had 5 switches swapped. It has been running for 1-2 month without problems and suddenly starts to do it...
It's a WS-8-150-AC
MAC Address ec:13:b2:d1:13:70
Firmware Version 1.4.7
Board Rev F
Site 2 has got WS-6-MINI installed. Has been running for 2-3 weeks without problems and suddenly starts to randomly reboot.
MAC Address ec:13:b2:91:6b:a6
Firmware Version 1.4.7
Board Rev F
For both sites it started out of the blue the other day. We have had really sunny days (cold weather), and that's why I suspect the solar inverters maybe to generate some kind of noise what ever, since both sites has solar systems installed probably generating full power because of the sun...

User avatar
jjonsson
Associate
Associate
 
Posts: 337
Joined: Wed Nov 05, 2014 12:30 pm
Location: Denmark
Has thanked: 37 times
Been thanked: 65 times

Re: Switch rebooting periodically......

Sun Dec 03, 2017 5:57 pm

Here's uptime graph for site A with WS-8-150-AC switch.....

WS-8-150-AC uptime 01.png


Start week 34 we started to see problems with the switch rebooting. It was swapped (switch no. 3), but problems was still there, switch rebooted randomly, Again in week 40 a new switch (no. 4) replaced the old.
In week 41 we found that grounding was not sufficient and fixed that. In week 41/42 we replaced again (switch no. 5) to make sure it was not the switch.
As you can see it ran without problems from week 42 til now where it suddenly started to reboot again....
That's why we don't expect the switch to be the problem, except if all 3 switches was from same batch (I actually don't know if they are ?)....
Switch 1 and 2 was replaced because of water in cable (Cable has been removed and new installed) 1½ years ago...
All cables has been checked for water, no water at all, cables are dry, so is switch/switchports etc....

Can you see from MAC address (above post) if switch is from bad batch ?

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Switch rebooting periodically......

Sun Dec 03, 2017 8:49 pm

Well you can test the WS-6 and WS-8-150-AC units to see if they have the 3.3V CAP issue which can cause reboots.

Setup on your bench

links "all" ports at 1G (no POE is required but does not affect the test) no real traffic is needed just links.

Let the suspected unit run all night and see it reboots, if it does not reboot it is fine.

I will post up a MAC range tomorrow that "could" be affected by this defective CAP issue but nothing made in the at least the past 6 months no longer is at risk.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

User avatar
jjonsson
Associate
Associate
 
Posts: 337
Joined: Wed Nov 05, 2014 12:30 pm
Location: Denmark
Has thanked: 37 times
Been thanked: 65 times

Re: Switch rebooting periodically......

Mon Dec 04, 2017 7:48 pm

sirhc wrote:I will post up a MAC range tomorrow that "could" be affected by this defective CAP issue but nothing made in the at least the past 6 months no longer is at risk.

I'll wait for that list and check....

MyThoughts
Member
 
Posts: 15
Joined: Tue Jun 09, 2015 3:30 pm
Location: Somewhere, Earth
Has thanked: 2 times
Been thanked: 1 time

Re: Switch rebooting periodically......

Sat Dec 23, 2017 9:48 pm

We have over the years had numerous WS-12-250-AC that reboot randomly as well.
Many times we have just replaced them with newer revisions. We are starting want to know what is going on as prior to gigabit based equipment we used 24V PoE midspan smart control devices from a competitor. We have only had one failure from those products in over 10 years of operating them and it was the flash on the unit that failed. We still have a few places where they are still in operation.

We have quite a few Netonix 12 port PoE switches deployed (>20) and the rate of replacement due to odd not easily explained issues is starting to bother us. I believe we have replaced around 6 units do to unexplained issues.

Just today we had another switch in operation less then 1.5 years randomly reboot.
The switch in question is a RevD and was recently upgraded this past Wednesday from v1.3.9 to v1.4.8.
Upgrading from v1.3.9 to v1.4.8 did solve one issue we were seeing, and none of our other switches that were upgraded to v1.4.8 have experienced any reboots so we are not sure if it is a coincidence or not.

Nothing looks out of the ordinary on the unit, board temp is 46, cpu is 57, and phy is 64. Voltages all check out, CPU usage is <30% and average throughput is around 300 Mbps. it has 175W of equipment that it is powering.

It is powered from a online double conversion UPS and when the issue has occurred only the switch (and PoE devices on the switch) go out, all the other equipment stays online. Logs from the WS-12-250-AC just report a cold boot.

If there is a known issue with certain boards do to a bad run I would definitely like to see a list of affected MAC addresses.

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Switch rebooting periodically......

Sat Dec 23, 2017 10:30 pm

Well since you upgraded to latest version see what happens.

v1.3.8 is 2+ year old firmware.

Of course the log is blank when it reboots as the log is kept in memory which is erased on reboot.

If you want to retain logs you use log servers.
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

MyThoughts
Member
 
Posts: 15
Joined: Tue Jun 09, 2015 3:30 pm
Location: Somewhere, Earth
Has thanked: 2 times
Been thanked: 1 time

Re: Switch rebooting periodically......

Sun Dec 24, 2017 6:27 am

Reboot issue occurred again ~1 hr ago, at of course 4AM.

The reboot issue has only occurred since upgrading to v1.4.8.

I have other switches that were upgraded a few days ago to v1.4.8 and they do not experience this reboot issue.

Aside from a coincidence is there anything that could be causing this behavior?

Otherwise I will have to toss this unit in the pile of other Netonix switches with unexplainable service impacting issues and replace it.

We have shutoff one of our backup AF24 units that is powered by this switch to see if lower heat/power usage have any affect.

Next
Return to Hardware and software issues

Who is online

Users browsing this forum: No registered users and 24 guests