USB NIC problem, going down and up -> Trying VLAN?

[raffe]

EDIT: OK, changing the subject, trying VLAN. See post 6...


Well, something seems to be wrong. I don't know if it has something to with the IP-number change (see https://forum.opnsense.org/index.php?topic=9344.0 ). I have

Code: [Select]

Name           OPNsense.localdomain
Versions          OPNsense 18.7-i386
FreeBSD           11.1-RELEASE-p11
OpenSSL           1.0.2o 27 Mar 2018
CPU Type      Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz (4 cores)
CPU usage        Load average 0.05, 0.05, 0.01
Uptime           1 days 17:24:29
State table size  0 % ( 573/346000 )
MBUF Usage   5 % ( 1536/26368 )
Memory usage      6 % ( 233/3465 MB )
SWAP usage   0 % ( 0/8192 MB )
Disk usage   1% / [ufs] (899M/106G)
I have installed OpenVPN (same as in https://docs.opnsense.org/manual/how-tos/sslvpn_client.html as I followed it, but I don't use 2FA.), I have activated NetFlow locally and have these plugins installed:
os-arp-scan (installed)   1.1   37.7KiB   Get all peers connected to a local network   
os-dyndns (installed)   1.8   134KiB   Dynamic DNS Support

I am connected with OpenVPN to OpnSense, and this happens from time to time (I am pinging the NAS that is on LAN):

Code: [Select]

...
...
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
Reply from 192.168.222.247: bytes=32 time=2ms TTL=63
...
...
In System: Log Files: General I see this at the time

Code: [Select]

Aug 7 10:45:57 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (dynamicraffe.botz.com): (Success) No Change In IP Address
Aug 7 10:45:57 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS: updating cache file /var/cache/dyndns_wan_dynamicraffe.botz.com_1.cache: 155.5.223.16
Aug 7 10:45:55 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (raffetest.botz.com): (Success) No Change In IP Address
Aug 7 10:45:55 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS: updating cache file /var/cache/dyndns_wan_raffetest.botz.com_0.cache: 155.5.223.16
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv6 default route
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway '155.5.223.97'
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 155.5.223.97
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: no IPv6 default gateway set, assuming wan
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
Aug 7 10:45:52 opnsense: /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
Aug 7 10:45:51 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS (dynamicraffe.botz.com): (Success) No Change In IP Address
Aug 7 10:45:51 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: updating cache file /var/cache/dyndns_wan_dynamicraffe.botz.com_1.cache: 155.5.223.16
Aug 7 10:45:49 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS (raffetest.botz.com): (Success) No Change In IP Address
Aug 7 10:45:49 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: updating cache file /var/cache/dyndns_wan_raffetest.botz.com_0.cache: 155.5.223.16
Aug 7 10:45:46 opnsense: /usr/local/etc/rc.newwanip: Interface '' is disabled or empty, nothing to do.
Aug 7 10:45:46 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpns1'
Aug 7 10:45:45 kernel: ovpns1: link state changed to UP
Aug 7 10:45:45 kernel: ovpns1: link state changed to DOWN
Aug 7 10:45:45 opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '155.5.223.97'
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 155.5.223.97
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 155.5.223.16) (interface: WAN[wan]) (real interface: ue0).
Aug 7 10:45:44 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ue0'
Aug 7 10:45:43 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface wan
Aug 7 10:45:43 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for wan
Aug 7 10:45:43 opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway 155.5.223.97.
Aug 7 10:45:43 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
Aug 7 10:45:43 kernel: ue0: link state changed to UP
Aug 7 10:45:43 kernel: ue0: link state changed to DOWN
Aug 7 10:31:08 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (dynamicraffe.botz.com): (Success) No Change In IP Address
...
...
If I in System: Log Files: General search for "ue0: link state changed to DOWN" I see

Code: [Select]

Aug 8 10:37:25 kernel: ue0: link state changed to DOWN
Aug 8 10:34:37 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:34:29 kernel: ue0: link state changed to DOWN
Aug 8 10:25:20 kernel: ue0: link state changed to DOWN
Aug 8 08:15:19 kernel: ue0: link state changed to DOWN
Aug 7 10:45:43 kernel: ue0: link state changed to DOWN
Aug 7 10:30:29 kernel: ue0: link state changed to DOWN
Aug 7 10:30:29 kernel: ue0: link state changed to DOWN
Aug 7 10:30:29 kernel: ue0: link state changed to DOWN
Aug 7 10:08:00 kernel: ue0: link state changed to DOWN
Aug 7 10:05:49 kernel: ue0: link state changed to DOWN
Aug 7 10:05:24 kernel: ue0: link state changed to DOWN
Aug 7 10:05:24 kernel: ue0: link state changed to DOWN
Aug 7 09:51:53 kernel: ue0: link state changed to DOWN
Aug 7 09:23:42 kernel: ue0: link state changed to DOWN
Aug 7 09:22:53 kernel: ue0: link state changed to DOWN
Aug 7 09:22:53 kernel: ue0: link state changed to DOWN
Aug 7 09:22:53 kernel: ue0: link state changed to DOWN
Aug 7 09:22:53 kernel: ue0: link state changed to DOWN
Aug 7 08:36:57 kernel: ue0: link state changed to DOWN
Aug 6 20:40:59 kernel: ue0: link state changed to DOWN
Aug 6 20:39:48 kernel: ue0: link state changed to DOWN
Aug 6 20:37:27 kernel: ue0: link state changed to DOWN
...
...
The NIC is a J5 JUE130 (https://en.j5create.com/products/jue130?variant=10610940932 ) and should have a AX88179 chipset (that is found here https://www.freebsd.org/releases/11.1R/hardware.html#ethernet ). "dmesg | grep AX" gives

Code: [Select]

ugen0.2: <ASIX Elec. AX88179> at usbus0

I use a USB NIC as the mini PC only have one NIC on board. I have done the same with IPcop for five years without problems, but it have a Startech USB31000SW (https://www.startech.com/se/en/Networking-IO/usb-network-adapters/USB-3-to-Gigabit-Ethernet-NIC-Network-Adapter~USB31000SW ) also with AX88179 chipset. Maybe that NIC is better?

I am sorry to say that I don't know much about FreeBSD, so I wonder if anyone of you could help me on how to start with finding the error? I don't even know how to see more than one page of logs at System: Log Files: General...

[bartjsmit]

For around the same cost, you can purchase a TP-Link SG108e managed switch and trunk the necessary VLAN's to OPNsense.

Just a thought

Bart...

[raffe]

I just saw that I had an unassigned ovpns1 interface 00:00:00:00:00:00 - XEROX CORPORATION

So I have now assigned it as OPT1 interface (opt1, ovpns1). Could this have something to do with my problems?

@Bart: If this will not get better, I may need to think about your idea. I know nothing about VLANs though I guess 802.1Q VLAN is the way to go, if I have understood it correctly. Do you think a TP-link TL-SG105E would also work? I see they have it in a shop very close by.

[guest18611]

Hello,

I can't help you very much, but I can say that I have OPNsense 18.7 successfully installed with this USB to Ethernet adapter https://www.amazon.com/dp/B00MYT481C

I also tried this one with AX88179 about 3 Months ago with no success https://www.amazon.com/dp/B00MYTSN18

Caution, it is only a 10/100 Mbps Adapter but for me it is ok, because I only have 100Mbps on the WAN line 

BTW: I have assigned the ovpns1 without any option except the description. You could try that also but I don't think that is the problem.

[raffe]

@Raccoon: Thanks for your reply. I have now tried with switching USB NICs between IPcop and OPNsense. So now IPcop has the new J5 JUE130 and OPNsense has the older Startech USB31000SW. Lets see if it OPNsense likes that Startech NIC better... Otherwise maybe I need to go that scary, mysterious and difficult muddy path towards the world of VLAN 

[bartjsmit]

Don't fear the VLAN ;-)

The Register has a gentle introduction: https://www.theregister.co.uk/2017/06/30/vlans_at_20/

In a nutshell; you trunk as many VLAN's as you want to use to OPNsense on a tagged port, and add untagged ports to the switch for all the bits of kit that you want to connect to each zone.

TL-SG105E is fine too, but the price per port is a bit higher than for the 108.

Bart...

[raffe]

Sooo... I tried VLAN. As ordered (  ) I bought a TP-Link SG108e managed switch, but I think I may be missing some knowledge to get it working.

I set up the TP-Link like this


And OpnSense like this


As I understand it, the difference between VLAN and PVID is that:
VLAN = Are the VLAN domains ID:s assignet to the port.
PVID = (Port VLAN ID) is the default VLAN id assigned to frames coming to the port.

LAN is connected to port 3
WAN is connected to port 6
I connect the re0 to port 1 or 2, and save all settings, but after that I can't connect to OpnSense any more. So I can't make any new firewall rules if that is what I need to do.

So I think I have done something wrong with my VLAN settings. Have I maybe done something wrong with choosing "Tagged" and "Untagged" ports?

Should port 1 and 2 be "Tagged" ports because OpnSense will "Tagging" the packets? And all other ports should be "Untagged"? Or is it the other way around  ? 

And should that be done in the settings for VLAN 1, 222 and 666? Or only 222 and 666? I think maybe only 222 and 666.

I am asking because I not home now, and will not be until next weekend, so now I can only plan and ponder on how I will make this work. Help me, Obi-Wan Kenobi. You're my only hope...

[bartjsmit]

Okay, if you want port 3,4, and 5 on VLAN 222 (LAN) and port 6,7, and 8 on VLAN 666 (WAN) with OPNsense on one of the remaining ports (let's say port 1), you need these assignments:

VLAN 222 name LAN
port 1 - tagged
port 2 - not member
port 3 - untagged
port 4 - untagged
port 5 - untagged
port 6 - not member
port 7 - not member
port 8 - not member

Click Add/Modify

VLAN 666 name WAN
port 1 - tagged
port 2 - not member
port 3 - not member
port 4 - not member
port 5 - not member
port 6 - untagged
port 7 - untagged
port 8 - untagged

Click Add/Modify

On the OPNsense console add VLAN 222 and assign the interface to be LAN. From one of the devices on port 3, 4, or 5 log into the GUI and add a VLAN under interfaces, other types, VLAN, add. Pick the ethernet interface as the parent and assign tag 666, description WAN. In interfaces, assignments set the VLAN interface to be the WAN.

Sit back, pour yourself a beverage of choice and accept the congratulations from friends and family.

Bart...

[raffe]

Thank you Bart!

I think I have it all set up almost right now, it is close, but still no cigar.

The OpnSense PC is a VivoPC VM62 https://www.asus.com/Mini-PCs/VivoPC_VM62/

Ethernet NIC is
dmesg | grep Realtek

Code: [Select]

re0: <Realtek PCIe GBE Family Controller> port 0xe000-0xe0ff mem 0xf7d00000-0xf7d00fff,0xf0000000-0xf0003fff irq 18 at device 0.0 on pci3dmesg | grep RTL

Code: [Select]

rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 3 on miibus0
My switch is set up like this
It should be just like Bart wrote, but I have two tagged trunk ports = ports 1 & 2, not just port 1 (I'm planing on reinstalling the IPcop PC with OpnSense and let it use port 2):



I have connected the cables to the switch like this


My OpnSense has these VLANs


NOTE!! -->It works with using VLAN 222 to LAN with this setup. Like this everything works well, all users can surf and so on.


But if I try to change also WAN via VLAN like this


Connection via LAN still works to OpnSense, but WAN never gets any IP from IPS DHCP so nobody can reach Internet. It is like the connection from VLAN666 on re0 don't communicate with ports 6, 7 & 8 on the switch.

[bartjsmit]

Can you ping IPCop from the OPNsense WAN? Does your ISP give you a public IP address over DHCP? If it is an RFC 1918 range, you will have to untick 'block private networks' from your WAN connection.

Also, is any traffic from the ISP device making it to OPNsense? Disconnect ue0 and IPCop temporarily and check for denied log entries on the WAN and/or do a packet trace. You can also use port 6 or 7 to mirror port 8 and run a workstation with Wireshark in promiscuous mode to see what's going on.

If there is any traffic, then the problem is not your VLAN config. Start from layer 1 and work your way up the stack :-)

Bart...