Messages

I'm having the same problem and tried this fix on both ends with no change. IPsec still maxes out around 30mbit. Did you have to do anything else to resolve this?

The traffic is going through the firewall for sure. As for specifics, I think I'm looking for clarification...

- outbound NAT rule: on LAN interface, src LAN Net, dst <ip> port 25, NAT address <external service ip>

This seemed to have too broad of a scope. I made it like this:

-outbound NAT rule: on LAN interface, src Single host or Network <internal ip/32 of exchange server>, dst <internal ip/32 of spam filter server>, translation/target <virtual public IP>

Did I do any of that incorrectly? I'd like all traffic between the two that touches the router to appear as though it is from the external IP. When I enable this rule, it makes the traffic appear as though it's coming from the actual private IP instead of the gateway, but that's still not what I need.

- fw rule on LAN: src address <external service ip>, dst <ip> port 25

I'm sorry, but this is where I really started to get confused. I don't understand this rule or why it would be needed. Also not sure which IP <external service ip> is referring to. Can you explain in a bit more detail please?

Thanks!!

Thank you for the quick response. I tried this but it's still showing the router's LAN IP in the email server logs.

Maybe I'm doing it incorrectly... Any other detail you can provide?

Hi,

I recently switched from a SonicWall router to an OPNsense appliance. Behind this appliance I have a multi-tenant Exchange server and a VM running Proxmox Mail Gateway spam filter. When someone in the multi-tenant Exchange server emails anyone else that the spam filter is filtering email for, obviously reflection comes into play and it works. However, the sender IP address detected by the spam filter is the OPNsense appliance's LAN IP instead of the WAN IP:

Mar 27 15:05:22 mx01 postfix/smtpd[12870]: D43C632D238: client=localhost.localdomain[127.0.0.1], orig_client=unknown[192.168.7.1]

With the SonicWall, the orig_client was correctly detected as the WAN IP. This is causing problems with some domains with enforced SPF.

I have OPNsense 19.1.4 set up with multi-wan on a small appliance. It has been in place and functioning great for quite a while, but last night it went offline and I wasn't on-site this morning so I walked users through power cycling the router and everything came back up.

I'm looking through the logs and the gateway log is the only one with relevant information:

Aug 2 08:15:54 dpinger: GATEWAY ALARM: WANGW (Addr: 8.8.8.8 Alarm: 0 RTT: 11001ms RTTd: 101ms Loss: 15%)
Aug 2 08:15:54 dpinger: WANGW 8.8.8.8: Clear latency 11001us stddev 101us loss 15%
Aug 2 08:15:45 dpinger: GATEWAY ALARM: WAN2_GWv4 (Addr: 8.8.4.4 Alarm: 0 RTT: 20396ms RTTd: 2472ms Loss: 13%)
Aug 2 08:15:45 dpinger: WAN2_GWv4 8.8.4.4: Clear latency 20396us stddev 2472us loss 13%
Aug 2 08:15:21 dpinger: WANGW 8.8.8.8: sendto error: 64
Aug 2 08:15:20 dpinger: WANGW 8.8.8.8: sendto error: 64
Aug 2 08:15:18 dpinger: GATEWAY ALARM: WAN2_GWv4 (Addr: 8.8.4.4 Alarm: 1 RTT: 0ms RTTd: 0ms Loss: 100%)
Aug 2 08:15:18 dpinger: WAN2_GWv4 8.8.4.4: Alarm latency 0us stddev 0us loss 100%
Aug 2 08:15:18 dpinger: GATEWAY ALARM: WANGW (Addr: 8.8.8.8 Alarm: 1 RTT: 0ms RTTd: 0ms Loss: 100%)
Aug 2 08:15:18 dpinger: WANGW 8.8.8.8: Alarm latency 0us stddev 0us loss 100%
Aug 2 08:15:16 dpinger: send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 8.8.4.4 bind_addr 98.191.85.167 identifier "WAN2_GWv4 "
Aug 2 08:15:16 dpinger: send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 8.8.8.8 bind_addr 216.255.247.46 identifier "WANGW "

All of these entries are from right after the router booted back up though. I've looked through:
https://docs.opnsense.org/manual/logging.html

There aren't any relevant entries anywhere from before it went down that I could find. Any idea where I should look to see what happened?

Thanks!

Someone? Anyone?

I have a very basic LAN/WAN configuration with static WAN address and a few ipsec vpn connections. I started investigating this one location because of poor VPN performance.

I did the "old ping MTU test" and found some interesting results that I'm not able to replicate anywhere else.

If I run this command while behind my opnsense: ping 8.8.8.8 -f -l 1473
I get the following: "Packet needs to be fragmented but DF set."

If I run this command while behind my opnsense: ping 8.8.8.8 -f -l 1472
I get the following: "Request timed out."

I don't begin to get replies until I get to: ping 8.8.8.8 -f -l 512
512 bytes and below reply.

I've put a laptop in place of my opnsense appliance with the static IP and it works as expected. Packets under 1473 bytes all reply. Can anyone shed some light on this? I'm running version 19.1.4.

Thanks!

When packets from 20 to 120 are entering enc0 device the Firewall is good. If 120 can reach 20, everything good. But when 20 can't access 120 and packets going to enc0, you'll see something in ipsec log or dropped packets.

While enabling public access to the webgui, I went ahead and updated opnsense and rebooted the 120.0 router. After the update and reboot, everything is now working properly.

I think the initial problem was that I did not have the LAN rule to route ipsec traffic via default gateway. I'm still not sure where that is documented, but thank you for sharing!

I have no clue what caused the latest one-way traffic issue, but it looks like either updating or rebooting cleared that up.

Thanks again for all of your help!

This looks good, then problem is on other side ...

Since I cannot connect to the 120.0 subnet without getting on a computer in that location, I have scheduled some time with a user tomorrow morning. Any suggestions of what specifically I should look for/at?

Again, connections work one-way. Users on the 120.0 subnet can browse to the server shares on the 20.0 network. It seems like connections initiated from the 120.0 subnet can be routed back, but something is wrong where connections initiated from the 20.0 subnet are not routed to the VPN correctly.

Come to think of it, I don't see any reason why I shouldn't just enable WAN access to the router while I'm troubleshooting tomorrow. But again, I'd love some suggestions of what to look for. It's a very basic setup in that location. Single WAN with static IP, single LAN, no port forwards or anything... just a single ipsec vpn.

Go to console and type:

tcpdump -n -i enc0 net 10.128.120.0/24

Then restart the ping and post the output

Here's the output:

~ # tcpdump -n -i enc0 net 10.128.120.0/24
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 262144                                              bytes
09:06:47.724605 (authentic,confidential): SPI 0xcc3a2824: IP 10.128.120.194.5365                                             2 > 10.128.20.32.161:  GetRequest(63)  .1.3.6.1.2.1.25.3.2.1.5.1 .1.3.6.1.2.1.25                                             .3.5.1.1.1 .1.3.6.1.2.1.25.3.5.1.2.1
09:06:50.178783 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.1                                             28.120.1: ICMP echo request, id 1, seq 534, length 40
09:06:54.931364 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 535, length 40
09:06:59.931996 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 536, length 40
09:07:04.928269 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 537, length 40

And here's the output with the verbose switch:
https://ufile.io/yqgwn

There's some other traffic in there before the ping packets...

Screenshot where both counters are non zero plz

https://imgur.com/a/POaMgMy

When you ping from 20 to 120 .. does the counter go up in ipsec status page?

It goes up a little whenever I refresh without pinging anything. I tried to tell if there was a difference while pinging, but it seemed the same... maybe the keep-alive is sending that traffic over the tunnel?

I asked for Ipsec logs. If you can reach one of them it's only a small issue.

Screenshot Ipsec connection status

Here's the IPsec connection status page:
https://imgur.com/a/99sGa8c

Here is part of the ipsec logs:
https://ufile.io/c2wcr

I hope it's something super simple and stupid that I missed. Users in the 120.0 location can browse to the server shares in the 20.0 location, so connectivity is working one way. Thanks!

Maybe that packet capture wasn't too helpful. These might be better. Again, I don't know what I'm looking at in wireshark... any help is greatly appreciated!

(1) is where I pinged the gateway on the 120.0 subnet, which did not reply. Then I pinged the gateway on the 121.0 subnet, which did reply.

(2) is where I was attempting to browse to the gateway's web interface on the 120.0 subnet, then the 121.0 subnet.

https://ufile.io/px5r3
https://ufile.io/t6bem

Anyone?