v1.5.22 Bug Reports and Comments

[josh@belwave.com]

I also cannot see connected mikroitks via ROMON behind the netonix.

Mikrotik to Netonix as a trunk port(with or without LAG, issue exists on both senarios), each netonix port has an access vlan.

https://pastebin.com/U35EGbXf

I also have multiple a point to point OSPF sessions using /31s that have stopped working.
- Update 12/7/24 16:06 CST Broadcast OSPF isn't working either.
- - Downgrade to 1.5.14 resolves both issues

[sirhc]

I also cannot see connected mikroitks via ROMON behind the netonix.

Mikrotik to Netonix as a trunk port(with or without LAG, issue exists on both senarios), each netonix port has an access vlan.

https://pastebin.com/U35EGbXf

I also have multiple a point to point OSPF sessions using /31s that have stopped working.
- Update 12/7/24 16:06 CST Broadcast OSPF isn't working either.
- - Downgrade to 1.5.14 resolves both issues

My entire WISP is OSPF and I see no issues?

My entire setup of my network is shown here in detail

Netonix Forums • View topic - This works well for us

[josh@belwave.com]

Good to hear you have no issues. However, I have downgraded all my devices back to 1.5.14 and everything is working again.

[sirhc]

Good to hear you have no issues. However, I have downgraded all my devices back to 1.5.14 and everything is working again.

I would suggest experimenting on one issue at a time on a small segment.

I would try turning services off in the switch, increasing MTU, things like that.

Give me something to work with.

But I would say a LARGE portion of our users use Mikrotik and have not heard this one yet?

I am glad and eager to help or do what we can, but I have nothing to really work with here.

If you insist on using v1.5.14 MAKE SURE you use Access Control List to prevent hacks.

Maybe a diagram showing what switch model, which ports go where, and so on.

The more info the better

[Flo]

We are still struggling with the issue having all PPPoE discovery packets (MAC protocol: 8863) being filtered on SFP ports.

I did some more debugging with the following findings on Friday:
- PPPoE client PADI packets send out the network are no longer received from PPPoE server
- changing SFP module brand does not resolve this issue
- unplug / re-plug SFP modules does not resolve this issue
- soft-reboot WS-250-AC does not resolve this issue
- hard-reboot WS-250-AC does not resolve this issue
- increasing the port MTU from 1528 to 1556 does not resolve this issue
- disable "Storm Control": Broadcast, Multicast, Unicast filters to "None" does not resolve this issue
- disable Discovery protocols does not resolve this issue

Downgrading to v1.5.16 does immediately resolve the issue.

If you have PPPoE discovery working on latest v1.5.22 via SFP links please share details on your setup.

[sirhc]

Anyone using PPPoE PLEASE help Flo and us resolve this issue.

I'll give a free Netonix T-Shirt and a $200 hundred dollar credit voucher for the Netonix web store for the one that helps solve this issue.

Otherwise Flow if not solved within a week or so well set up a lap to figure it out. Problem is I don't use PPPoE at my WISP so I can't readily test it nor am I that familiar with it so Stephen and I will have to do some reading.

FOR NOW MAKE SURE YOUR USING THE ACCESS CONTROL LIST TO PROTECT YOURSELF FROM BEING HACKED.

Sorry for the not so fast solution on this.

[mayheart]

Are you using "trunk" option under VLANs?

Here is the setup I'm using on all pages.

-> Ports
Isolate/DHCP Snooping enabled on AP ports. MTU is 1528 for APs
SFP uplink port, 9000 MTU, only stats enabled.

-> VLANs
no port trunking, manually tagging all VLANs in use. PPPoE VLAN is tagged on uplink port, untagged facing APs. tagged facing BHs to other towers.

-> QoS
disabled

-> STP
disabled

-> Device/Config
Pause/Loop is disabled
All discovery options are disabled

Doing a packet capture, I'm still seeing the usual PPPoE discovery from both the client and server. I migrated another Netonix that has a SFP uplink to 1.5.22 late last night, no issues reported.

Would it help if I send one of my device configs to Stephenhttps://forum.netonix.com/memberlist.php?mode=viewprofile&u=7920

[josh@belwave.com]

Good to hear you have no issues. However, I have downgraded all my devices back to 1.5.14 and everything is working again.

I would suggest experimenting on one issue at a time on a small segment.

I would try turning services off in the switch, increasing MTU, things like that.

Give me something to work with.

But I would say a LARGE portion of our users use Mikrotik and have not heard this one yet?

I am glad and eager to help or do what we can, but I have nothing to really work with here.

If you insist on using v1.5.14 MAKE SURE you use Access Control List to prevent hacks.

Maybe a diagram showing what switch model, which ports go where, and so on.

The more info the better

After downgrade to 1.5.14 I can no longer login to the web gui(i had reset the password on 1.5.22). I did ssh into the unit and reset the user/pass on the config and that has regained web access.
The 2 locations that I noticed the issues with OSPF were indeed using sfp modules.

Just upgraded one of the switches that have an ethernet uplink and neighbors + OSPF is working properly. I suspect that possibly the issue is with the sfp module or some forwarding of multicast that isnt getting to the sfp? Just a suspicion at this point. I'm going to keep digging.

[sirhc]

We did completely re-write the code dealing with the SFPs as there were so many issues with SFP modules not coming back up after upgrades or warm reboots.

So, it is possible that brand of SFP does not like the new code.

Sadly, very few SFP manufacturers follow the spec.

When working with the NEW WS3 10G SFP modules we found numerous brands that did not properly report to the switch that they were copper and the switch though they were fiber, and fiber does not step down. So even though the copper SFP module supported 10/100/1000/10000 it always showed any link as 10G. There was nothing we could do in code to correct this as this is handled on the SFP control module which has its own code.

SFP modules basically have 2 parts. A control module that talks to the switch and the PHY, and of course a PHY.

This little issue wasted almost a week of coding until the SOC development team contacted us and said "OH YEA, YOU MIGHT FIND THIS ISSUE, NO WORK AROUND".

[josh@belwave.com]

We did completely re-write the code dealing with the SFPs as there were so many issues with SFP modules not coming back up after upgrades or warm reboots.

So, it is possible that brand of SFP does not like the new code.

Sadly, very few SFP manufacturers follow the spec.

When working with the NEW WS3 10G SFP modules we found numerous brands that did not properly report to the switch that they were copper and the switch though they were fiber, and fiber does not step down. So even though the copper SFP module supported 10/100/1000/10000 it always showed any link as 10G. There was nothing we could do in code to correct this as this is handled on the SFP control module which has its own code.

SFP modules basically have 2 parts. A control module that talks to the switch and the PHY, and of course a PHY.

This little issue wasted almost a week of coding until the SOC development team contacted us and said "OH YEA, YOU MIGHT FIND THIS ISSUE, NO WORK AROUND".

I understand that fully, had numerous issues with mikrotik ros7 and modules... still do. I do have a router/netonix setup with both copper ethernet and fiber sfp. and can confirm the issue with neightbors romon and OSPF only exist on the sfp modules.
router module: Mikrotik S-85DLC05D
Netonix: Mikrotik S-85DLC05D