Messages

If anyone lands here, I was having a similar issue with DNS as mentioned here[1]. Putting a switch between the Starlink Ethernet adapter and the OPNsense box fixed it, but I have no idea why.

[1] https://forum.netgate.com/topic/174105/starlink-problem-with-sg2440-22-05/38

Casper, what were the steps you took to get DNS working?

The port forward seems to be working now.

I went into Firewall -> Settings -> Advanced and changed the settings highlighted in the attached screenshot.  I still don't understand the gateway concept, but will do some research.  Disabling reply-to didn't work (not sure if that was the option you were referring to Franco).

Thanks for the help!  I wouldn't have thought to look for NAT settings under Advanced.

EDIT 1: I will reinstall from scratch when 18.1 comes out and try again.

Code Select Expand

root@OPNsense:~ # cat /tmp/rules.debug
set ruleset-optimization basic
set optimization normal
set timeout { adaptive.start 0, adaptive.end 0 }
set limit states 814000
set limit src-nodes 814000

# System aliases
loopback = "{ lo0 }"
lan = "{ igb0 }"
wan = "{ igb1 }"

# SSH Lockout Table
table <sshlockout> persist
table <webConfiguratorlockout> persist
# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"
table <bogonsv6> persist file "/usr/local/etc/bogonsv6"

# User Aliases

# Plugins tables

set loginterface igb0

set skip on pfsync0

scrub on $lan all   
scrub on $wan all   

no nat proto carp
no rdr proto carp

# Outbound NAT rules (automatic)

# Subnets to NAT
tonatsubnets  = "{ 127.0.0.0/8 192.168.1.0/24 }"
nat  on $wan from $tonatsubnets to any port 500 -> 184.9.150.155/32  static-port
nat  on $wan from $tonatsubnets to any -> 184.9.150.155/32 port 1024:65535 

# Anti lockout, prevent redirects for protected ports to this interface ip
no rdr on igb0 proto tcp from any to ( igb0 ) port { 443 80 22 }

# NAT Inbound Redirects
rdr on igb1 inet proto tcp from any to (igb1) port 10022 -> 192.168.1.102
# Reflection redirect
rdr on igb0 inet proto tcp from any to (igb1) port 10022 -> 192.168.1.102
no nat on igb0 proto tcp from igb0 to 192.168.1.102 port 10022
nat on igb0 proto tcp from 192.168.1.0/24 to 192.168.1.102 port 10022 -> 192.168.1.1 port 1024:65535


antispoof log for igb0
antispoof log for igb1
#pass in  log quick on lo0 inet6 from {any} to {any} label "Pass all loopback IPv6"
#block in  log quick inet6 from {any} to {any} label "Block all IPv6"
block in  log inet from {any} to {any} label "Default deny rule"
block in  log inet6 from {any} to {any} label "Default deny rule"
pass in  log quick inet6 proto ipv6-icmp from {any} to {any} icmp6-type {1,2,135,136} keep state label "IPv6 requirements (ICMP)"
pass out log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {129,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {ff02::/16} to {fe80::/10} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
block in  log quick inet proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet6 proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet proto {tcp udp}  from {any} to {any}  port {0}
block in  log quick inet6 proto {tcp udp}  from {any} to {any}  port {0}
block in log quick proto carp from {(self)} to {any}
pass in  log quick proto carp from {any} to {any}
block in log quick proto tcp from {<sshlockout>} to {(self)}  port {22} label "sshlockout"
block in log quick proto tcp from {<webConfiguratorlockout>} to {(self)}  port {443} label "webConfiguratorlockout"
block in  log quick from {<virusprot>} to {any} label "virusprot overload table"
#block in log quick on lo0 from {<bogons>} to {any} label "block bogon IPv4 networks from loopback"
#block in log quick on lo0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from loopback"
#block in log quick on lo0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from loopback"
#block in log quick on igb0 from {<bogons>} to {any} label "block bogon IPv4 networks from LAN"
#block in log quick on igb0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from LAN"
#block in log quick on igb0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from LAN"
block in log quick on igb1 from {<bogons>} to {any} label "block bogon IPv4 networks from WAN"
block in log quick on igb1 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from WAN"
block in log quick on igb1 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from WAN"
pass in log quick on igb0 proto udp from {any}  port {68} to {255.255.255.255}  port {67} label "allow access to DHCP server"
pass in log quick on igb0 proto udp from {any}  port {68} to {(self)}  port {67} label "allow access to DHCP server"
pass out log quick on igb0 proto udp from {(self)}  port {67} to {any}  port {68} label "allow access to DHCP server"
pass in  log quick on igb0 inet6 proto udp from {fe80::/10} to {fe80::/10,ff02::/16}  port {546} label "allow access to DHCPv6 server on LAN"
pass in  log quick on igb0 inet6 proto udp from {fe80::/10} to {ff02::/16}  port {547} label "allow access to DHCPv6 server on LAN"
pass in  log quick on igb0 inet6 proto udp from {ff02::/16} to {fe80::/10}  port {547} label "allow access to DHCPv6 server on LAN"
pass in log quick on igb0 inet6 proto udp from {fe80::/10} to {(self)}  port {546} label "allow access to DHCPv6 server on LAN"
pass out log quick on igb0 inet6 proto udp from {(self)}  port {547} to {fe80::/10} label "allow access to DHCPv6 server on LAN"
pass in log on igb1 proto udp from {any}  port {67} to {any}  port {68} label "allow DHCP client on WAN"
pass out log on igb1 proto udp from {any}  port {68} to {any}  port {67} label "allow DHCP client on WAN"
pass in  log quick on lo0 from {any} to {any} label "pass loopback"
pass out log from {any} to {any} keep state allow-opts label "let out anything from firewall host itself"
pass in log quick on igb0 proto tcp from {any} to {(self)}  port {443 80 22} keep state label "anti-lockout rule"
pass out log  route-to ( igb1 184.9.144.1 ) from {igb1} to {!(igb1:network)} keep state allow-opts label "let out anything from firewall host itself"
pass in  quick on igb0 inet from {(igb0:network)} to {any} label "USER_RULE: Default allow LAN to any rule"
pass in  quick on igb0 inet6 from {(igb0:network)} to {any} label "USER_RULE: Default allow LAN IPv6 to any rule"
pass in  quick on igb1 inet proto tcp from {any} to {192.168.1.102}  port {10022} label "USER_RULE: NAT "

[reply-to]

Are you testing from a network attached to WAN or from the Internet?

Because "reply-to" will not reroute the traffic to your test client in WAN. The traffic goes to your upstream WAN gateway, and if that one doesn't route it back it looks like it doesn't work. This is a safeguard for multi-wan. You can disable it in the firewall advanced settings.

pass in  quick on igb1 reply-to ( igb1 184.9.144.1 )  inet proto tcp from {any} to {192.168.1.102}  port {10022} label "USER_RULE: NAT "



Cheers,
Franco

Hi Franco,

I am testing from a machine on my LAN.

Playing around in the settings, I noticed the Gateway with an IP different from the WAN.  I wasn't sure what it was or its purpose.  Later today, I'll try disabling it and retesting.

I don't have a multi-WAN.  Is the Gateway something that gets added by default in an install?  I'm curious if maybe I misconfigured something on the install.  I've installed once about 15 months ago and have been doing all the updates.  After a long time trying to get this to work, I "reset to factory settings" in an effort to remove variables.

Thanks in advance.

Hi Denis,

I attached a screenshot of the Port Forward page in my first post.

Can you explain further why it should be the Local Address and not the WAN?  My thinking was the packet was coming from the internet, so the destination when it hits the router should be the WAN Address.

I've been having a lot of trouble getting a port forward working.  My goal is to forward SSH to a desktop.

Related to the SSH connection, I'm using keys, and it works within the LAN.  I've changed the port to 10022 just to rule out any issues related to the default 22.

igb0 is my LAN, igb1 is my WAN.

Attached is a screenshot of my Port Forward page.

I've tried "catching" the connection as I try to SSH in, but I don't see it in the Normal View of the Log Files.  Not sure how to continue to debug this as I'm just getting a connection timeout.  Any help is appreciated.

EDIT 1: canyouseeme.org is reporting 10022 open.  It was reporting 22 was open as I was trying that port (and is now closed).

(desktop) ssh -vvv user@184.9.150.155
OpenSSH_7.5p1, LibreSSL 2.6.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolving "184.9.150.155" port 10022
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 184.9.150.155 [184.9.150.155] port 10022.
debug1: connect to address 184.9.150.155 port 10022: Operation timed out

root@OPNsense:/tmp # cat rules.debug
set ruleset-optimization basic
set optimization normal
set timeout { adaptive.start 0, adaptive.end 0 }
set limit states 814000
set limit src-nodes 814000

# System aliases
loopback = "{ lo0 }"
lan = "{ igb0 }"
wan = "{ igb1 }"

# SSH Lockout Table
table <sshlockout> persist
table <webConfiguratorlockout> persist
# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"
table <bogonsv6> persist file "/usr/local/etc/bogonsv6"

# User Aliases

# Plugins tables

set loginterface igb0

set skip on pfsync0

scrub on $lan all   
scrub on $wan all   

no nat proto carp
no rdr proto carp

# Outbound NAT rules (automatic)

# Subnets to NAT
tonatsubnets  = "{ 127.0.0.0/8 192.168.1.0/24 }"
nat  on $wan from $tonatsubnets to any port 500 -> 184.9.150.155/32  static-port
nat  on $wan from $tonatsubnets to any -> 184.9.150.155/32 port 1024:65535 

# Anti lockout, prevent redirects for protected ports to this interface ip
no rdr on igb0 proto tcp from any to ( igb0 ) port { 443 80 }

# NAT Inbound Redirects
rdr on igb1 inet proto tcp from any to (igb1) port 10022 -> 192.168.1.102
# Reflection redirect
rdr on igb0 inet proto tcp from any to (igb1) port 10022 -> 192.168.1.102

antispoof log for igb0
antispoof log for igb1
#pass in  log quick on lo0 inet6 from {any} to {any} label "Pass all loopback IPv6"
#block in  log quick inet6 from {any} to {any} label "Block all IPv6"
block in  log inet from {any} to {any} label "Default deny rule"
block in  log inet6 from {any} to {any} label "Default deny rule"
pass in  log quick inet6 proto ipv6-icmp from {any} to {any} icmp6-type {1,2,135,136} keep state label "IPv6 requirements (ICMP)"
pass out log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {129,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {fe80::/10} to {fe80::/10,ff02::/16} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
pass in log quick inet6 proto ipv6-icmp from {ff02::/16} to {fe80::/10} icmp6-type {128,133,134,135,136} keep state label "IPv6 requirements (ICMP)"
block in  log quick inet proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet6 proto {tcp udp}  from {any}  port {0} to {any}
block in  log quick inet proto {tcp udp}  from {any} to {any}  port {0}
block in  log quick inet6 proto {tcp udp}  from {any} to {any}  port {0}
block in log quick proto carp from {(self)} to {any}
pass in  log quick proto carp from {any} to {any}
block in log quick proto tcp from {<sshlockout>} to {(self)}  port {22} label "sshlockout"
block in log quick proto tcp from {<webConfiguratorlockout>} to {(self)}  port {443} label "webConfiguratorlockout"
block in  log quick from {<virusprot>} to {any} label "virusprot overload table"
#block in log quick on lo0 from {<bogons>} to {any} label "block bogon IPv4 networks from loopback"
#block in log quick on lo0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from loopback"
#block in log quick on lo0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from loopback"
#block in log quick on igb0 from {<bogons>} to {any} label "block bogon IPv4 networks from LAN"
#block in log quick on igb0 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from LAN"
#block in log quick on igb0 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from LAN"
block in log quick on igb1 from {<bogons>} to {any} label "block bogon IPv4 networks from WAN"
block in log quick on igb1 from {<bogonsv6>} to {any} label "block bogon IPv6 networks from WAN"
block in log quick on igb1 from {10.0.0.0/8,127.0.0.0/8,100.64.0.0/10,172.16.0.0/12,192.168.0.0/16,fc00::/7} to {any} label "Block private networks from WAN"
pass in log quick on igb0 proto udp from {any}  port {68} to {255.255.255.255}  port {67} label "allow access to DHCP server"
pass in log quick on igb0 proto udp from {any}  port {68} to {(self)}  port {67} label "allow access to DHCP server"
pass out log quick on igb0 proto udp from {(self)}  port {67} to {any}  port {68} label "allow access to DHCP server"
pass in  log quick on igb0 inet6 proto udp from {fe80::/10} to {fe80::/10,ff02::/16}  port {546} label "allow access to DHCPv6 server on LAN"
pass in  log quick on igb0 inet6 proto udp from {fe80::/10} to {ff02::/16}  port {547} label "allow access to DHCPv6 server on LAN"
pass in  log quick on igb0 inet6 proto udp from {ff02::/16} to {fe80::/10}  port {547} label "allow access to DHCPv6 server on LAN"
pass in log quick on igb0 inet6 proto udp from {fe80::/10} to {(self)}  port {546} label "allow access to DHCPv6 server on LAN"
pass out log quick on igb0 inet6 proto udp from {(self)}  port {547} to {fe80::/10} label "allow access to DHCPv6 server on LAN"
pass in log on igb1 proto udp from {any}  port {67} to {any}  port {68} label "allow DHCP client on WAN"
pass out log on igb1 proto udp from {any}  port {68} to {any}  port {67} label "allow DHCP client on WAN"
pass in  log quick on lo0 from {any} to {any} label "pass loopback"
pass out log from {any} to {any} keep state allow-opts label "let out anything from firewall host itself"
pass in log quick on igb0 proto tcp from {any} to {(self)}  port {443 80} keep state label "anti-lockout rule"
pass out log  route-to ( igb1 184.9.144.1 ) from {igb1} to {!(igb1:network)} keep state allow-opts label "let out anything from firewall host itself"
pass in  quick on igb0 inet from {(igb0:network)} to {any} label "USER_RULE: Default allow LAN to any rule"
pass in  quick on igb0 inet6 from {(igb0:network)} to {any} label "USER_RULE: Default allow LAN IPv6 to any rule"
pass in  quick on igb1 reply-to ( igb1 184.9.144.1 )  inet proto tcp from {any} to {192.168.1.102}  port {10022} label "USER_RULE: NAT "

I think the card is ok.  It works in the other machine.

I ran the card without the riser in the box with errors and still got them, so I think the riser is ok too.

I picked up a switch and started using the ports on the motherboard for WAN/LAN and everything's been good for a while.  I'm guessing the PCI slot is somehow broken.

The card I have is a dual port and I'm using both.

I think the only other PCI card I have is a USB 2.0 card, but I can give it a shot.

I'm probably going to end up getting a switch and using the two ports on the motherboard.

The PCI slot on the motherboard with the timeouts is 3.3v.  The card is PCI-X, but from what I've read, this card should work in any PCI slot, but at a slower speed.

http://forums.untangle.com/hardware/31398-urgent-intel-pwla8494mt-pro-quad-port.html

Is that incorrect?

Memtest reported no errors.

It was easier to run the card without the riser than to try the riser in a different machine.  Unfortunately, same results.

Good catch depnding on config hot to tough may totally be within specification. Even the power regulators for your CPU are rated far above what you can physically touch. That doesn't rule out it might be a contributing factor but if it works in another system then thats a good indication its not the issue.

Can you perhaps try the card in a bench machine or another desktop system with that 90º riser card? its possible there is a damaged trace of the riser is simply faulty (fails continuity).

Otherwise I would be interested in memtest results. and a bios reset. which a flash should do for you depending on board. so that we may be able to scratch off power saving/other features from interrupting the bus in an odd way.

Sorry for late reply.

I flashed the BIOS (was already on the latest), but no improvement.

I'm running memtest right now (win for IPMI).  It should complete a pass within 80 minutes and I'll update when its done.

While that's running, I think I can get the riser on the board to test.

The NIC is working in the other machine.  It is still running very hot.

The CPU temps were in the high 30's.

I'm pretty sure this is just a hardware issue.  I even had trouble trying to start up memtest.  I'm going to flash the BIOS and then start from scratch again.

Thanks for everyone's help.  If this actually starts working again, I'll post it.

I have a similar type case, except backwards.
Superserver with a Atom C2758F in it, four SSD's (two SATA, one SATA-DOM and a PCI-E 4X card.
Only has one fan, next to the one in the PSU, and keeps everything in normal temperature ranges.

If that NIC really is getting so hot, it's the card and not the case/cooling.
The NIC doesn't have a heatsink, let alone active cooling, as far as I can tell?
Then it shouldn't overheat.

Correct, it has no heat sink.

hm have you tried doing a memtest? I'm not actually fantastic at networking. but im actually very good with hardware if BSD is indicating watchdog errors than the fault may actually lie with the overall system stability. The previous edition working could have simply been coincidence. Can you take this machine offline to run extended diagnostics? Are you able to get any thermal measurements from the CPU? Does SMART indicate drive failure?

I'm starting to think that it could be a hardware issue with the nic card.  I have this setup in a 1U case (link below).  I'm using SuperMicro's 90 degree PCI piece.

I noticed the case itself was really hot, so I popped the cover off and the card was too hot to touch.  The heatsink on the CPU was cool.  I've been running a room fan into it (uncovered).  This seems to lessen the errors, but definitely does not stop it.  Is it possible I already burned the card up?

Tonight I'll put the card in the other machine and see if I still get the timeouts and if its running hot.  I will try memtest/SMART/temp tests before switching over as well.

Thanks for the help.

http://www.supermicro.com/products/chassis/1U/504/SC504-203.cfm

I wasn't able to get the HardenedBSD image to work.  I get to the point where it puts me in a prompt "mountroot>".   Maybe I'm writing it to the USB incorrectly?

Code Select Expand

gzcat FILE.img.xz | dd of=/dev/da0 bs=64k && sync

With the 10.2 upgrade, I am getting the same timeout issue.

pfSense 2.2.4 is giving me the same issue.  I've been bouncing back and forth between 2 machines for my router.  It must've been working on the other one (an old Dell optiplex).  OPNsense was working there too.