OPNsense Forum
Archive => 18.7 Legacy Series => Topic started by: Adam.P on January 14, 2019, 05:00:01 pm
-
I have a customer with 3 locations. Everything was initially setup with OPNsense 17 with IPSec VPN setup between all three locations. Everything worked perfectly. A second WAN was added to one location. Since then I can only communicate with devices one way - devices in the other two offices can ping everything in every location. If I am in the office with 2 WAN connections, traffic will not route through the VPN. I can only communicate with devices on that local network.
I read in release notes that there were some routing fixes, so i've performed all updates to 18.7.10 and still am having the same problem. Anyone have any ideas?
-
Anyone?
-
Am I posting this in the wrong place or not including enough information?
-
You should really post some screenshots of Gateways, Tiering and Rules. Also outbound Nat. And dont forget to set a static route for Ipsec remote IP
-
You should really post some screenshots of Gateways, Tiering and Rules. Also outbound Nat. And dont forget to set a static route for Ipsec remote IP
Thank you for the response.
Here are some screenshots: https://imgur.com/a/4ZKeRug
It's a pretty basic setup. I setup a single LAN/WAN, followed this article to setup the VPN:
https://wiki.opnsense.org/manual/how-tos/ipsec-s2s.html
Then followed this article to setup multi-wan:
https://wiki.opnsense.org/manual/how-tos/multiwan.html
Please let me know if I can provide anything else.
Thank you!
-
And dont forget to set a static route for Ipsec remote IP
How do I create this static route? IPSec isn't an option when creating a route.
I already have these routes in the routing table which were automatically created by ipsec:
ipv4 10.128.120.0/24 173.8.42.14 US 97 1500 em1 WAN
ipv4 10.128.121.0/24 173.8.42.14 US 5 1500 em1 WAN
I don't think they're being followed though. I tried running trace route and it just reaches max hops and stops.
-
For Ipsec network you have to create an Accept Rule without Gateway above your routing rules
-
For Ipsec network you have to create an Accept Rule without Gateway above your routing rules
You're referring to this rule, right?
https://imgur.com/a/YSSxs3j
-
No, for Multi WAN you set in LAN rules tab like in the official docs
-
No, for Multi WAN you set in LAN rules tab like in the official docs
Are the wiki's I linked above not official? I didn't see any mention of this in those documents... Can you link me to said document?
I tried creating this rule and it did not appear to help:
https://imgur.com/a/22WSwvt
"Remote_Networks" is an alias to 10.128.120/24 and 10.128.121/24
-
Please post screenshots of LAN rules tab
-
Please post screenshots of LAN rules tab
That's exactly what the above screenshot is. I'll post the URL again: https://imgur.com/a/22WSwvt
Thank you!
-
Local DNS rule should be at the top, rest is fine
-
Local DNS rule should be at the top, rest is fine
I went ahead and moved it to directly below the anti-lockout rule. Thanks for that!
I just did some more testing after doing a states reset and found that I can communicate with the 10.128.121.0/24 network but still cannot communicate with the 10.128.120.0/24 network.
I'm not sure when that started working. Honestly, I've been doing most of my testing with the 120 network and assumed 121 also wouldn't work still.
I'm confused now. How would one work but not the other?
-
Ipsec problem? You can do a packet capture on enc inteface and check if packets are leaving correctly. Also check ipsec logs
-
Sorry it took so long to follow up on this. I don't see anything abnormal in the IPsec logs. I'm not sure how to read the packet captures, but here's a packet capture taken from the ipsec interface.
https://ufile.io/5ro7v
Please let me know if that's helpful or if there's any other info I can provide to get to the bottom of this. I'm stumped! Thanks in advance!
-
Anyone?
-
Maybe that packet capture wasn't too helpful. These might be better. Again, I don't know what I'm looking at in wireshark... any help is greatly appreciated!
(1) is where I pinged the gateway on the 120.0 subnet, which did not reply. Then I pinged the gateway on the 121.0 subnet, which did reply.
(2) is where I was attempting to browse to the gateway's web interface on the 120.0 subnet, then the 121.0 subnet.
https://ufile.io/px5r3
https://ufile.io/t6bem
-
Hello there,
The problem for me at least, I don't have a similar network to test this configuration, so I can't offer much help. Anyway, if you aren't getting any help here, try raising a Github issue instead.
Regards
-
I asked for Ipsec logs. If you can reach one of them it's only a small issue.
Screenshot Ipsec connection status
-
I asked for Ipsec logs. If you can reach one of them it's only a small issue.
Screenshot Ipsec connection status
Here's the IPsec connection status page:
https://imgur.com/a/99sGa8c
Here is part of the ipsec logs:
https://ufile.io/c2wcr
I hope it's something super simple and stupid that I missed. Users in the 120.0 location can browse to the server shares in the 20.0 location, so connectivity is working one way. Thanks!
-
When you ping from 20 to 120 .. does the counter go up in ipsec status page?
-
When you ping from 20 to 120 .. does the counter go up in ipsec status page?
It goes up a little whenever I refresh without pinging anything. I tried to tell if there was a difference while pinging, but it seemed the same... maybe the keep-alive is sending that traffic over the tunnel?
-
Screenshot where both counters are non zero plz
-
Screenshot where both counters are non zero plz
https://imgur.com/a/POaMgMy
-
Go to console and type:
tcpdump -n -i enc0 net 10.128.120.0/24
Then restart the ping and post the output
-
Go to console and type:
tcpdump -n -i enc0 net 10.128.120.0/24
Then restart the ping and post the output
Here's the output:
~ # tcpdump -n -i enc0 net 10.128.120.0/24
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 262144 bytes
09:06:47.724605 (authentic,confidential): SPI 0xcc3a2824: IP 10.128.120.194.5365 2 > 10.128.20.32.161: GetRequest(63) .1.3.6.1.2.1.25.3.2.1.5.1 .1.3.6.1.2.1.25 .3.5.1.1.1 .1.3.6.1.2.1.25.3.5.1.2.1
09:06:50.178783 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.1 28.120.1: ICMP echo request, id 1, seq 534, length 40
09:06:54.931364 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 535, length 40
09:06:59.931996 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 536, length 40
09:07:04.928269 (authentic,confidential): SPI 0xce0c18d7: IP 10.128.20.10 > 10.128.120.1: ICMP echo request, id 1, seq 537, length 40
And here's the output with the verbose switch:
https://ufile.io/yqgwn
There's some other traffic in there before the ping packets...
-
This looks good, then problem is on other side ...
-
This looks good, then problem is on other side ...
Since I cannot connect to the 120.0 subnet without getting on a computer in that location, I have scheduled some time with a user tomorrow morning. Any suggestions of what specifically I should look for/at?
Again, connections work one-way. Users on the 120.0 subnet can browse to the server shares on the 20.0 network. It seems like connections initiated from the 120.0 subnet can be routed back, but something is wrong where connections initiated from the 20.0 subnet are not routed to the VPN correctly.
Come to think of it, I don't see any reason why I shouldn't just enable WAN access to the router while I'm troubleshooting tomorrow. But again, I'd love some suggestions of what to look for. It's a very basic setup in that location. Single WAN with static IP, single LAN, no port forwards or anything... just a single ipsec vpn.
-
When packets from 20 to 120 are entering enc0 device the Firewall is good. If 120 can reach 20, everything good. But when 20 can't access 120 and packets going to enc0, you'll see something in ipsec log or dropped packets.
-
When packets from 20 to 120 are entering enc0 device the Firewall is good. If 120 can reach 20, everything good. But when 20 can't access 120 and packets going to enc0, you'll see something in ipsec log or dropped packets.
While enabling public access to the webgui, I went ahead and updated opnsense and rebooted the 120.0 router. After the update and reboot, everything is now working properly.
I think the initial problem was that I did not have the LAN rule to route ipsec traffic via default gateway. I'm still not sure where that is documented, but thank you for sharing!
I have no clue what caused the latest one-way traffic issue, but it looks like either updating or rebooting cleared that up.
Thanks again for all of your help!