Messages

The Intel I210 based NIC arrived and was fitted.  The WAN is now stable.

Moral of the story is, don't rely on a RealTek NIC, even if it worked before. If you have to, use the vendor driver.

Fedora worked with the a Realtek USB NIC, so I tried the same in my OPNSense router and it failed.

You have broken two rules :
- USB Networking for critical services on your network.
- RealTek NIC in combination with FreeBSD.

That's TWO BIG DON'Ts at the same time! ;)

Thanks.  I agree with the Realtek one - they seem rather poorly supported.

However, the USB NIC was just for testing.  My permanent NIC is internal, but just so happens to also be using a chipset manufactured by the devil :-)

It's being replaced by an Intel I210-AT based card tomorrow.

[don't]

Have you tried with the MAC Address spoofing?
My UK fibre provider works perfectly well without it and I get no connection with it.

Just to clarify - are you saying that you don't spoof the MAC and it works?  I'm with YouFibre and everyone says I have to spoof. I tried with and without and I could only get an IP address when spoofing while using the built-in Realtek driver.  Now that I've swapped to the vendor driver, and it's working, I realised I hadn't spoofed the MAC address.  I'm naturally reluctant to go a tweak again!

I would start looking for clues in two places: message buffer and system log which is /var/log/system/latest.log
Perhaps a live tail (tail -f) whilst renewing the dhcp lease from the ISP.

Thanks for the suggestion of looking in the logs.  That got me thinking - it might not be firewall/routing, but lower level network issues.

Fedora worked with the a Realtek USB NIC, so I tried the same in my OPNSense router and it failed.  The logs showed the NIC bouncing.  A quick Google showed me that the default Realtek driver isn't the best with FreeBSD and a vendor one is available.  I installed that, and the OPNSense router's internal NIC for the WAN side (also a Realtek) came up straight away and gave me full speed.

I've now ordered an Intel based NIC for it, so that I can keep away from Realtek.

What's strange is that this internal Realtek NIC worked reliably with my previous provider.  They were using an OpenReach ONT though (made by Nokia apparently) while the current provider uses an Adtran ONT.  There must be some incompatibility somewhere.

This is what happens when someone who is comfortable with Linux dabbles with FreeBSD :-)

Have you tried with the MAC Address spoofing?
My UK fibre provider works perfectly well without it and I get no connection with it.

I have to spoof the MAC address, otherwise I don't get an IP address from YouFibre's DHCP servers.

Have you gone to your WAN interface settings on OPNsense and disabled "Block private networks" and "Block bogon networks"? I would start there.

You can also try deleting and re-adding the WAN interface.

Thanks for the suggestions.  The two blocks were already off.  They need to be as the OPNSense router gets a 192.168.x.x address from the YouFibre router, in my current workaround.

I deleted and re-added the WAN interface, but that didn't help.

I also had some tunables I'd set for PPPoE of the previous provider and set those back to default, but that didn't help neither.

YouFibre is an UK ISP which provides routers which connect via DHCP (no PPPoE involved).

All Internet advice suggest that to use with your own router (OPNSense obviously), you need to ensure that the MAC address of the WAN side of your router matches what YouFibre expect, otherwise their DHCP server will refuse to issue you an IP address. Fortunately, their router has its MAC address printed on the base.

If I plug a Fedora laptop into the Optical Network Termination box and override the MAC address of the laptop with that taken from their router, then the connection comes up and a quick speed test shows a blistering 990Mbs on a 1Gbs package.  This shows that the principle works.

However, if I move the Ethernet cable to my OPNSense router and configure the same MAC address, I'm issued an IP address and given a default gateway via DHCP, but there's no further traffic.

I should add that the OPNSense router has been in service for a couple of years now, but the previous ISP used PPPoE.  I simply changed the WAN interface's settings from PPPoE to DHCP as suggested.

I can use the built-in packet sniffer to watch ICMP packets leave the router, but nothing returns.

I've even created a temporary any-any rule on the firewall just in-case I misconfigured something there.

I'm currently online by plugging the OPNSense router (with the MAC address reverted to default) into the LAN port of the provided YouFibre router, and am still managing 900Mbs or so, but I'd rather take the provided router out of the loop if possible.

Has anyone come across a similar situation, either with YouFibre or other DHCP and MAC address based ISPs?

[Advanced Features]

I've just figured this out, thanks to a post by @davidsenk, which pointed me to https://forum.opnsense.org/index.php?topic=15900 which discusses reply-to within rules.

In OpnSense's GUI I edited the rule and expanded the Advanced Features section. Within this section is the reply-to menu, which I set to disable.

After saving the rule and reloading the firewall, everything seems to be working.

I'm trying to allow DNS queries to my router, but not from the WAN interface.

To do this I've created a floating rule which allows TCP & UDP port 53 in.  However, this also allows it on the WAN interface, which I don't want.

I therefore added the WAN interface to the rule's 'Interface' field and selected 'Invert'.  This blocked DNS on all interfaces, not just the WAN.

While I've worked around this by reverting to all interfaces and setting the 'Source' to an alias consisting of local networks, I'd like to know why this doesn't work by inverting the interface.

Below is an extract from rules.debug which I've grepped on the interface (re0) and edited to remove the NAT entries:

Code Select Expand


# block in log quick on re0 inet from {<bogons>} to {any} label "a785cde4d07ef9d5492b2752d6f3674c" # Block bogon IPv4 networks from ONT
# block in log quick on re0 inet6 from {<bogonsv6>} to {any} label "1abb3c3b8584670c042452464f78d963" # Block bogon IPv6 networks from ONT
# block in log quick on re0 inet from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16} to {any} label "b6e046ea0da3e8b5479bb57aa34db5b1" # Block private networks from ONT
# block in log quick on re0 inet6 from {fc00::/7} to {any} label "fb42f48e27b4fd4647cd998434aea4d7" # Block private networks from ONT
pass out route-to ( re0 <next hop>) from {(re0)} to {!(re0:network)} keep state allow-opts label "f6dc4c3fe096989ac6d4a2c85cd16c64" # let out anything from firewall host itself (force gw)
pass in quick on  ! re0 reply-to ( re0 <next hop> ) inet proto {tcp udp} from {any} to {(self)} port {53} keep state label "f7314d8b59355b1c287b12cb88a291bd" # Allow incoming local DNS queries


As you can see, there are no block rules before it hits my DNS rule (the last one listed above).  Does anyone have any ideas why this fails?

Thanks.

I have multiple VLAN interfaces on my new OpnSense install.

I have noticed that those which are configured to issue addresses using DHCP (ISC DHCP v4 - not tried anything else) don't show up when I run `dig` or `drill` to ask the router to resolve it's own FQDN (as set up in System > Settings > General).  Only the interfaces which don't use DHCP are listed.

Code Select Expand

gareth@admin:~$ dig gw1.example.org

; <<>> DiG 9.18.24-1-Debian <<>> gw1.example.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60738
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 8, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;gw1.example.org. IN A

;; ANSWER SECTION:
gw1.example.org. 3600 IN A 172.29.11.1
gw1.example.org. 3600 IN A 172.29.5.1
gw1.example.org. 3600 IN A 172.29.10.1
gw1.example.org. 3600 IN A 172.29.7.1
gw1.example.org. 3600 IN A 172.29.15.1
gw1.example.org. 3600 IN A 172.29.4.1
gw1.example.org. 3600 IN A 172.29.1.1
gw1.example.org. 3600 IN A 192.168.10.197

;; Query time: 0 msec
;; SERVER: 172.29.1.1#53(172.29.1.1) (UDP)
;; WHEN: Mon Apr 15 07:31:30 BST 2024
;; MSG SIZE  rcvd: 185

Notice that 172.29.4.1 and 172.29.5.1 are listed.  After configuring those interfaces to use DHCP, I get:

Code Select Expand

gareth@admin:~$ dig gw1.example.org

; <<>> DiG 9.18.24-1-Debian <<>> gw1.example.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50886
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;gw1.example.org. IN A

;; ANSWER SECTION:
gw1.example.org. 3600 IN A 172.29.10.1
gw1.example.org. 3600 IN A 172.29.7.1
gw1.example.org. 3600 IN A 172.29.15.1
gw1.example.org. 3600 IN A 172.29.1.1
gw1.example.org. 3600 IN A 192.168.10.197
gw1.example.org. 3600 IN A 172.29.11.1

;; Query time: 0 msec
;; SERVER: 172.29.1.1#53(172.29.1.1) (UDP)
;; WHEN: Mon Apr 15 08:00:40 BST 2024
;; MSG SIZE  rcvd: 153

Notice that 172.29.4.1 and 172.29.5.1 are now missing.

Is there a valid reason for this?  Is it a setting which I've missed?  Or is it a bug?