Help with Intel X553 and 1G SFP

[Greg_E]

Reboot the computer or switch and/or power the computer/switch off, then insert the module. It's the old way of doing things and you aren't supposed to need to do this, but sometimes it makes a difference.

[shor0814]

Reboot the computer or switch and/or power the computer/switch off, then insert the module. It's the old way of doing things and you aren't supposed to need to do this, but sometimes it makes a difference.

I got it now, I was somehow thinking, "reboot the SFP??" but no, that makes sense to reboot the switch with the SFP inserted so it recognizes it during system initialization.

Thanks, and will try it next week, the ISP is coming out to look at why it isn't working.  I have to pay for their time, but that doesn't bother me to do so.

[shor0814]

Well, it was an interesting visit from the tech.  We pulled the SFP, swapped it into my UDM, and nothing, not a dang thing.  Put it back into his router, lights were red, no connection.  He grabbed another router, re-provisioned, the whole works, nothing.  The beauty of my install, they didn't document the switch I was attached to, which port number, nothing, I guess that is the price for being a first customer.

Knowing he couldn't do anything at this point, he escalated to his NOC and they sent out their network guy who managed to find my switch and port, it could really only be 1 of 2 cabinets and he just looked at the connected ports with no lights and reset the ports.  The router started working by the time he got to my house.

We talked about what I wanted to do, he was 100% certain it would work because they do some small business installs the same way.  Turns out, I was doing ok, just needed a VLAN, which was my guess all along.  Once we put the VLAN in and tried it, it went dead.  He went back to the switch and reset the port and back in business.  He will be diagnosing the issue, I should be able to plug and unplug the SFP without resetting the port.

For now, I am happy with where we are, on the flip side, I have ordered some FS.com transceivers coded for Intel.  In addition, the network guy thinks that every time I switch hardware the port gets locked up.  So once I have time I will add the VLAN to the OPNsense box, call him, and swap over.  He can reset the port if need be and help troubleshoot.

It was really outstanding service overall, just some things that didn't make sense and potentially something up with their gear.  So, no solution on the X553, but, too many irons in the fire to tackle it this moment so will work on that later.

I appreciate all of the help from everyone, just a combination of ISP issues, no VLAN, and not enough time.

[shor0814]

So I had some time today and decided to get back to this, I actually kind of regret it.

Since my last attempt I decided to get a static IP address, which should make a few things easier for me since the ISP allegedly doesn't need to have the MAC address anymore.

That said, here is what happened.

1. Connected the twinax cable to port ix0
2. Connected the fiber sfp module to port ix1

Ran ifconfig -v ix0 - showed the twinax connected
Ran ifconfig -v ix1 - showed nothing
Ran ifconfig -v on ix2 and ix3 and ix3 shows nonexistent interface while ix2 showed no module.

Digging around dmesg, showed that ix3 had an unsupported transceiver.  Now I have "hw.ix.unsupported_sfp=1" configured in my /boot/loader.conf.local so that shouldn't happen.  I unplugged all the transceivers and powerd off/on.

During startup and in dmesg, the issue with ix3 came up again and it is the only one with the unsupported message.  Odd since nothing is plugged in.  I attached the twinax to various ports and tried ifconfig -v until I found the one that showed the twinax attached.  Various times, the command would show the twinax while other times nothing.

I tried the same thing with the fiber and at some point, it actually showed the SFP module, brand, temperature, voltage, etc.

Thinking I figured it all out, I went forward with the configuration using the ix numbers that had the twinax and fiber attached.  I was able to get to the GUI to do some of the final configuration and at that point I configured what I thought to be the correct settings:

My ISP assigns a VLAN, so I configured a new VLAN with tag 85 in the settings as it should be.
Assigned the VLAN to the WAN interface
Set my static IP address and gateway in the WAN configuration.

I wasn't able to get an internet connection so I rebooted, thinking maybe the SFP needed to be in the port, etc.
On startup, I wasn't able to get to the GUI.
I went back to the SFP ports and did the same ifconfig -v ix? command and the ports that had the cables attached either showed nothing, or, in one case, the twinax showed on ix1 instead of ix0.

I moved interfaces around for the fiber, same issue, the port changed, I am able to get the module information, but each time, the interface moves. 

I have a suspicion that when the ix3 shows an unsupported module, it is causing some sort of interface renumber.  In fact, it may be that each time, a different interface is treated as ix3.  As I write this, I didn't check the MAC addresses to see if they move around with the interface numbers or not.  Unless of course Intel is dynamically setting the MAC addresses?

I am stumped and I can't reliably troubleshoot at this point.  I might go ahead and try pfSense tomorrow just for kicks unless someone has an idea what might be happening (other than flaky as hell hardware).

As for my configuration for the ISP, am I doing things correctly with the VLAN?

If this fails, I have a VEP4600 I might try.  Overkill for sure but I have a bit of a crunch to get something that has BGP support.  My Dream Machine supports BGP, but, there is so little memory it takes the machine down.

Thoughts?  Ideas are appreciated.