Netonix to Cisco LACP issue?

aaron@futuretk.com
Member
 
Posts: 20
Joined: Mon Feb 13, 2017 10:43 am
Has thanked: 0 time
Been thanked: 0 time

Netonix to Cisco LACP issue?

Thu Feb 22, 2018 4:48 pm

I'm troubleshooting an odd disconnect issue with one of 5 Netonix Switches in the same rack. The Netonix Switch has been replaced. The general behavior is that I loose connection to to the switch over the LACP bond and historically I've had a tech reboot the switch and it all comes right back up. Today I was near by and was able to connect directly to the switch to further diagnose.

Netonix WS-12-400-AC on 1.4.9 firmware and dual Fiber SFP modules
Cisco 6500 Switch with dual 16 port WS-X6416-GBIC cards and Cisco fiber GBIC modules

During the event the Cisco log showed simultaneous disconnects and reconnects of both ports facing the offending Netonix on roughly 20 second and 10 second intervals.

.Feb 22 19:33:33.457: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up
.Feb 22 19:33:33.533: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
Feb 22 19:33:33.461: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up
Feb 22 19:33:33.533: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
.Feb 22 19:33:52.385: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to down
.Feb 22 19:33:52.613: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to down
Feb 22 19:33:52.385: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to down
Feb 22 19:33:52.613: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to down
.Feb 22 19:34:03.189: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
.Feb 22 19:34:03.769: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up
Feb 22 19:34:03.185: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
Feb 22 19:34:03.765: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up
.Feb 22 19:34:22.509: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to down
.Feb 22 19:34:22.697: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to down
Feb 22 19:34:22.513: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to down
Feb 22 19:34:22.717: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to down

On the Netonix I showed both SFP ports up and connected at 1G, but the LACP status was Not Active on both ports. I disabled both SFP ports, clicked Save/Apply, enabled both SFP ports, clicked Save/Apply, and a few seconds later it came back up and started working again.

This is the Netonix logs from when I got up there and plugged my laptop into port 9, through when I unplugged it. There were no log entries in the previous 48 hours leading up to the event.

Feb 22 13:33:36 Port: link state changed to 'up' (10M-F) on port 9
Feb 22 13:33:36 STP: set port 9 to discarding
Feb 22 13:33:37 Port: link state changed to 'down' on port 9
Feb 22 13:33:40 Port: link state changed to 'up' (1G) on port 9
Feb 22 13:33:40 STP: set port 9 to discarding
Feb 22 13:33:43 STP: set port 9 to learning
Feb 22 13:33:43 STP: set port 9 to forwarding
Feb 22 13:34:11 UI: User 'admin' logged in from 10.1.1.222
Feb 22 13:34:54 UI: Configuration changed by admin (10.1.1.222)
Feb 22 13:34:54 UI: Port 13 Enable: changed from 'Enabled' to 'Disabled'
Feb 22 13:34:54 UI: Port 14 Enable: changed from 'Enabled' to 'Disabled'
Feb 22 13:35:09 monitor: restarting vtss_appl
Feb 22 13:35:11 STP: set port 1 to discarding
Feb 22 13:35:11 STP: set port 4 to discarding
Feb 22 13:35:11 STP: set port 9 to discarding
Feb 22 13:35:13 Port: link state changed to 'up' (1G) on port 14
Feb 22 13:35:13 Port: link state changed to 'up' (1G) on port 14
Feb 22 13:35:14 STP: set port 9 to learning
Feb 22 13:35:14 STP: set port 9 to forwarding
Feb 22 13:35:14 STP: set port 4 to learning
Feb 22 13:35:14 STP: set port 4 to forwarding
Feb 22 13:35:14 STP: set port 1 to learning
Feb 22 13:35:14 STP: set port 1 to forwarding
Feb 22 13:35:14 UI: Configuration changed by admin (10.1.1.222)
Feb 22 13:35:14 UI: Port 13 Enable: changed from 'Disabled' to 'Enabled'
Feb 22 13:35:14 UI: Port 14 Enable: changed from 'Disabled' to 'Enabled'
Feb 22 13:35:20 LACP: starting negotiation with partner 00-18-74-97-24-00
Feb 22 13:35:24 LACP: LACP changed state to Active on port 13 (key 4)
Feb 22 13:35:25 LACP: LACP changed state to Active on port 14 (key 4)
Feb 22 13:38:45 sSMTP[3209]: Sent mail for SharpSwitch4@futuretk.com (221 smtp.ellipse.net Mail Server ** ABUSE IS STRICTLY PROHIBITED **) uid=0 username=admin outbytes=7378
Feb 22 13:39:39 Port: link state changed to 'down' on port 9
Feb 22 13:39:39 STP: set port 9 to discarding

These are the Cisco Logs during the same time period of time.(Don't mind the 6 hours Time Zone setting difference)

.Feb 22 19:34:57.117: %LINK-3-UPDOWN: Interface GigabitEthernet8/4, changed state to down
.Feb 22 19:34:57.381: %LINK-3-UPDOWN: Interface GigabitEthernet9/4, changed state to down
Feb 22 19:34:57.116: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/4, changed state to down
Feb 22 19:34:57.380: %LINK-SP-3-UPDOWN: Interface GigabitEthernet9/4, changed state to down
.Feb 22 19:35:17.026: %LINK-3-UPDOWN: Interface GigabitEthernet8/4, changed state to up
.Feb 22 19:35:17.946: %LINK-3-UPDOWN: Interface GigabitEthernet9/4, changed state to up
Feb 22 19:35:16.974: %LINK-SP-3-UPDOWN: Interface GigabitEthernet8/4, changed state to up
Feb 22 19:35:17.950: %LINK-SP-3-UPDOWN: Interface GigabitEthernet9/4, changed state to up
.Feb 22 19:35:21.970: %LINK-3-UPDOWN: Interface Port-channel4, changed state to up
.Feb 22 19:35:22.094: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel4, changed state to up
.Feb 22 19:35:22.098: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
Feb 22 19:35:22.070: %LINK-SP-3-UPDOWN: Interface Port-channel4, changed state to up
.Feb 22 19:35:22.778: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up
Feb 22 19:35:22.074: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet8/4, changed state to up
Feb 22 19:35:22.202: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel4, changed state to up
Feb 22 19:35:22.754: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet9/4, changed state to up

I have attached a quick drawing of how the two devices are connected. All 5 Netonix switches are connected in the same way. SFP13 connects to a port on Module 8, and SFP14 connects to a port on Module 9. This issue is only impacting this one switch in the stack.

I've also attached a screen shots of the Status, Ports, and LAG tabs in the Netonix.

Thank you
Aaron Clark
Attachments
LAG tab.PNG
Ports Tab.PNG
Status Tab.PNG
Netonix to Cisco.PNG

User avatar
sirhc
Employee
Employee
 
Posts: 7347
Joined: Tue Apr 08, 2014 3:48 pm
Location: Lancaster, PA
Has thanked: 1597 times
Been thanked: 1317 times

Re: Netonix to Cisco LACP issue?

Thu Feb 22, 2018 6:08 pm

I would try:

1) Try setting the speed / duplex to AUTO on the SFP modules preferably in both Netonix and Cisco

2) swap out the SFP modules in the switch and if no love swap out the modules in the CISCO
Support is handled on the Forums not in Emails and PMs.
Before you ask a question use the Search function to see it has been answered before.
To do an Advanced Search click the magnifying glass in the Search Box.
To upload pictures click the Upload attachment link below the BLUE SUBMIT BUTTON.

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 27 guests