Topics

1

23.7 Legacy Series / connection issue on opnsense after cablemodem change

« on: October 12, 2023, 10:14:10 am »

I have a strange issue on one of our opnsense firewalls. And i don't know how to correctly set the title for this issue.

Because of an issue with a bad performance on the site the provider changed the old cablemodem to a new one. The modem provides the public ip to the opnsense via dhcp, so the WAN interface is configured as DHCP client (and not changed during the switch of the modem)

Since then the opnsense looses irregular the network connection. ping of the public and ping of the internal ip on the LAN interface are not possible anymore. the provider says they dont see a problem on there side.

Then 2-3 times powercycle the opnsense (its an apu2) and then it looks stable for hours.

I see in the log many entries like this:

Code: [Select]

<27>1 2023-10-06T23:29:10+02:00 cust-ch-xyz01-fw01.customer.intra dhclient 60785 - [meta sequenceId="1"] Bogus Host Name option 12: cpe_000db94ae4d4 (cpe_000db94ae4d4)
<13>1 2023-10-06T23:29:10+02:00 cust-ch-xyz01-fw01.customer.intra dhclient 56941 - [meta sequenceId="2"] Creating resolv.conf
but this log entries are already there before we switched to the new cablemodem. So i think this is not the issue? but perhaps there is also a solution to fix this?

This night, the connection was lost again around 02:20 o'clock. the opnsense is uptdating itself around thursday on 02:00 o'clock. the reboot was done around 02:17 o clock and as i look into the  system log and see that the wan interface (igb0) looses his connection and also lan (igb1) and then some scripts are triggered, which causes the loose of connections, and so on?

Because also i see something like:

Code: [Select]

<11>1 2023-10-12T02:20:13+02:00 cust-ch-xyz01-fw01.customer.intra opnsense 67423 - [meta sequenceId="71"] /usr/local/etc/rc.linkup: The command '/sbin/ifconfig 'igb1' inet '192.168.123.1'/'24'' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'
So perhaps some scripts are triggered get stuck?

Currently after a power cycle the opensense come up and then looses his connection and perhaps , some seconds later, it comes back again and is available via network. But it can loos it afterwards again. This morning we needed three

Im currently not at the location and there is no one with a serial cable so view on the console , when connection is lost, is not possible.

The almost complete log of today (until i write this) is attached here.

p.s i changed ip's and so on in the log for privacy.

2

Web Proxy Filtering and Caching / haproxy tcp mode passthrough to existing server not working

« on: January 31, 2023, 09:49:18 am »

Hi,

I searched the forum and read all the threads (with the tutorials) that i found about haproxy configuration, tried different approaches but nothing worked as expected. ssl connection always fails (ex. firefox SSL_ERROR_RX_RECORD_TOO_LONG) or when i try it with openssl s_client to check the certificate it looks like more , no certficate is given or it runs in non-ssl mode or something.

Currently i had a running setup with classic nat directly to a windows exchange 2016 server. Currently the server do the complete let's encrypt stuff by himself and this shoul'd not change. The server is in productive mode and is used internal and externaly. (split dns).

Because their is only one external ip i need to share this ip on port 443, so as i read haproxy is a good solution to solve this. So a secondary server with web services shoul'd be installed now behind the opnsense too.

So for this solution the let's encrypt stuff shoul'd not be happening on the opnsense, the server themselfs shoul'd be maintain the renewal, and so on. So for this the "tcp (layer 4) modes are the way to work with, or are i wrong?

Because the complete stuff on 80 and 443 is productive, i use 9080 and 9443 for testing purposes as ports on the firewall.

Below is my config (some stuff is defined but not used like the resolver currently) . i tried it last evening around 4-5 hours without sucess, so perhaps i oversea something that someone other, which had a working setup, see directly and i don't see it like the trees in the forest. ;-)

Code: [Select]

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    1
    hard-stop-after             60s
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.chksize                16384
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 debug
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats


# Resolver: dc-cust-01
resolvers 63d8cfcde3f718.78402437
    nameserver 192.168.99.11:53 192.168.99.11:53
    resolve_retries 3
    timeout resolve 1s
    timeout retry 1s



# Frontend: http ()
frontend http
    bind 192.168.99.1:9080 name 192.168.99.1:9080
    bind 1.x.x.x:9080 name 1.x.x.x:9080
    mode http
    option http-keep-alive
    default_backend ex-cust-01_http_backend
    # tuning options
    timeout client 30s

    # logging options
    option httplog
    # ACL: http_mail_customer_ch
    acl acl_63d834ead32456.82956086 hdr_sub(host) -i mail.customer.ch

    # ACTION: http_mail_customer_ch
    use_backend ex-cust-01_http_backend if acl_63d834ead32456.82956086

# Frontend: https_sni ()
frontend https_sni
    bind 192.168.99.1:9443 name 192.168.99.1:9443
    bind 1.x.x.x:9443 name 1.x.x.x:9443
    mode tcp
    default_backend ex-cust-01_https_backend
    # tuning options
    timeout client 30s

    # logging options
    option tcplog
    # ACL: ssl_hello
    acl acl_63d84150d59f08.49426761 req_ssl_hello_type 1
    # ACL: https_mail_customer_ch
    acl acl_63d83503a02259.11459738 req.ssl_sni -m sub -i mail.customer.ch

    # ACTION: tcp_request_inspect_delay
    # NOTE: actions with no ACLs/conditions will always match
    tcp-request inspect-delay 5s
    # ACTION: tcp_request_content_accept_ssl
    tcp-request content accept if acl_63d84150d59f08.49426761
    # ACTION: https_mail_customer_ch
    use_backend ex-cust-01_https_backend if acl_63d83503a02259.11459738
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-request set-header X-Real-IP %[src]

# Backend: ex-cust-01_https_backend (Exchange 2016 - HTTPS)
backend ex-cust-01_https_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server ex-cust-01_https 192.168.99.11:443 ssl verify none send-proxy-v2 check-send-proxy send-proxy-v2 check-send-proxy

# Backend (DISABLED): matrix-cust-01_https_backend (Matrix Chat HTTPS)

# Backend: ex-cust-01_http_backend (Exchange 2016 - HTTP)
backend ex-cust-01_http_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    # WARNING: pass through options below this line
    option forwardfor
    http-reuse safe
    server ex-cust-01_http 192.168.99.11:80

# Backend (DISABLED): matrix-cust-01_http_backend (Matrix Chat HTTP)

on the "ex-cust-01_https_backend" i also tried with and without ssl in the "server" line. the owa, and other services are only via ssl reachable on the server, port 80 always gets empty white pages expect the .well-known directory for let's encrypt.

3

22.7 Legacy Series / nat for some internal ip's is not working

« on: December 05, 2022, 11:52:48 am »

Hi,

I have the problem on some opnsense firewall's that when the firewall is restartet, that some yealink phones can't connect anymore to the pbx on the wan side.

The provider told me that three phones can't connect (i dont have access to the phones). When i now check in the firewall  and do a tcpdump i see that the affected phone's try to connect to the pbx. "SIP: REGISTER" is visible on the LAN and also on the WAN side.  They only thing that i see, that the three ip's are with their internal IP's visible on the tcpdump on the pppoe0 interface. Shoul'd that not be the public ip of the firewall as on the other phones?

I also had the same issue on another customer with an internal pbx which connect to the sip provider directly.

I use the default NAT settings. The firewall version is 22.7.9 commit b0c31af1a.

Update/Edit:

After some search i deleted the entries of the affected internal ip in the "Firewall: Diagnostics: States" view and then nat nat worked again. But why this happens and how can we prevent it?

4

German - Deutsch / One-to-One NAT OpenVPN über IPSEC will nicht mehr

« on: July 15, 2021, 04:07:51 pm »

Seit Heute Morgen, wieso auch immer, funktioniert de "One-to-One" NAT nicht mehr auf der einen Firewall.

Es ist so das Auf der Seite der Firewall das LAN 172.23.19.0/24 existiert, sowie die HomeOffice Leute per OpenVPN mit dem LAN 172.23.20.0/24 verbinden. 

Die Firewall wiederum macht zu einer Branchenlösung einen Site2Site Tunnel per IPSEC zum Hersteller der Branchenlösung 10.2.3.0/24. Die Clients im LAN verbinden so auf den RDP Server 10.2.3.10 was ohne Problem klappt. Da der IPSEC Tunnel jedoch nur 172.23.19.0/24 zu 10.2.3.0/24 durchstellt und 172.23.20.0/24 nicht mit gerouted werden kann, hatte ich hier ein wie in der Doku als Beispiel drin ein One-to-One NAT eingerichtet.

Nun ist die Frage, hatte das bisher nur durch Zufall funktioniert, oder liegt der Hund wo anders begraben?

Interface: IPsec
Type: BINAT
External Network: 172.23.19.0/24
Source: 172.23.20.0/24
Destination: 10.2.3.10/32   (Da es nur den RDP Server braucht, dachte ich das reicht)

5

21.1 Legacy Series / default deny rule matchs, and ignores wan rule

« on: July 01, 2021, 11:35:24 am »

I have a strange issue on one of our firewalls. i found some nearly similar threads here in the forum, but the solutions their don't work. But it looks like a double-nat asynchronious routing problem, but im'm not the expert here.

On the router of the provider, i had a port forwarding for ipsec and also https and ssh.  (no source ip restrictions possible)

On the firewall i have the rule to allow all traffic from the source ip of our office to the wan interface.

But when i try to connect from the office, the "default deny" rule matches and the traffic is dropped.

I tried with the advanced setting of the rule and the state type "sloppy" and "none" but this don't have any effect.

Internet <=> Provider Router, 192.168.1.1 <=> OPNsense Firewall 192.168.1.128

All IP's are fixed.  Netstat on the firewall told me the following: (The ip's i x-ed are vpn ipsec networks, which currently also not work, the 192.168.9.0 net is the guest wlan)

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS        igb0
10.x.x.x/24        192.168.1.1        US         igb0
127.0.0.1          link#5             UH          lo0
172.21.9.0/24      link#2             U          igb1
172.21.9.1         link#2             UHS         lo0
172.27.x.x/16      192.168.1.1        US         igb0
192.168.1.0/24     link#1             U          igb0
192.168.1.128      link#1             UHS         lo0
192.168.x.0/24     192.168.1.1        US         igb0
192.168.9.0/25     link#8             U      igb1_vla
192.168.9.1        link#8             UHS         lo0

A i was onsite to install the firewall on monday's the connections worked, but now not anymore without a change (until now which i tried with sloppy, and so on).

6

German - Deutsch / Double NAT Problematik mit LAN to DMZ

« on: August 08, 2019, 01:18:43 pm »

Ich habe das Problem das ich bei einem Setup nicht um ein Double NAT herum komme.

Auf der ersten Firewall (Sehr mininmal und kaum Einstellmöglichkeiten) habe ich ein DNAT und SNAT von Public IP auf virtuelle IP's der OPNsense eingerichtet. Die wiederum ein 1:1 NAT auf den Host in der DMZ macht. Das funktioniert soweit gut. Ich sehe in der DMZ die richtigen IP's der Clients auf dem Server aber auch wenn ich vom Server ins Internet verbinde, sehe ich die richtige zugewiesene fixe IP Adresse. Die OPNsense Firewall kennt in diesem Szenario die wirkliche Public IP gar nicht, da sie selber am WAN Interface mit Internen virtuellen IP's arbeitet die 1:1 genatted sind auf der ersten Firewall.

Nun ist das Problem aber das ich ein LAN auf der OPNsense habe und die auch auf gewisse Dienste zugreifen müssen und ich würde das ganze wenn möglich mit NAT lösen oder Routing lösen am LAN Interface. (DNS, etc. will ich nicht umbiegen) Braucht es da zwingend eine virtuelle IP der Public IP am LAN Interface, oder geht das auch ohne, das ich einfach eine NAT Regel einrichte, das wenn eine Anfrage vom LAN an die Public IP kommt, die Firewall ein NAT durchführt und das ganze so dann in die DMZ schickt und die DMZ über die FW das wieder zurück, da ich nicht möchte, das der Host der DMZ die LAN IP des Clients sieht.

Ich habe verschiedenes ausprobiert, aber das Gefühl das ich vor Lauter Bäumen den Wald nicht sehe, oder das gewünschte Setup so wirklich nicht geht.

7

19.1 Legacy Series / IPSEC Tunnel not working anymore

« on: March 13, 2019, 11:08:28 am »

We had the problem, that since our firewall updated to 19.1.4 our ipsec tunnel's don't work anymore as expected. We see on the GUI that all tunnels are up and on both sites we see status up on both phase. But we don't see any traffic through the tunnel. I can ping from my host on site A to the firewall or the server of site B and i don't see any traffic on Site B, i only see with tcpdump my ping request on the Firewall of site A but nothing more. We also rebooted the firewall but no effect.

Edit:
Site B ist still on version 19.1.2 and when i ping from Site B to Site A i see in tcpdump on interface enc0 ping ping request on site A and Site B. When i do a ping from Site A, i don't see this ping on enc0. Only on the LAN interface and then it goes to nowhere.