Messages

I also tried applying a mark to the traffic and that didn't do anything either.

I did a search and filtered on the destination ports I'm trying to move through the VPN.  I get connectivity but I never see the rules show up in the firewall log when I filter based on them.  I've confirmed the port is correct from my server and a connection is established (this if from a netstat on the server):

tcp        0      0 192.168.32.2:59276      x.x.x.x:25461    ESTABLISHED.

The application works but I never see the traffic flow through the VPN tunnel.  Here are my rules currently in pfTop that relate to this, I don't know if that helps:

pfTop: Up Rule 1-122/122, View: rules
RULE  ACTION   DIR LOG Q IF     PR        K     PKTS    BYTES   STATES   MAX INFO                                                                                                                                                                             
109  Pass     In  Log Q ix1_vl tcp       K        0        0        *       route-to ... inet from (ix1_vlan32) to any port = 826  flags S/SA                                                         
110  Pass     In  Log Q ix1_vl udp       K        0        0        *       route-to ... inet from (ix1_vlan32) to any port = 826                                                                     
111  Pass     In  Log Q ix1_vl tcp       K        0        0        *       route-to ... inet from (ix1_vlan32) to any port = 25461  flags S/SA                                                       
112  Pass     In  Log Q ix1_vl udp       K        0        0        *       route-to ... inet from (ix1_vlan32) to any port = 25461   

I do have the OpenVPN instance assigned to an interface.  I'm not sure what you are saying about the NAT and interface groups?

I played around with this a bit this morning.  I had the VPN client set not to pull routes.  Once I do that, I can get traffic through the VPN tunnel but it appears that it sends ALL traffic through, even though I have just the outbound nat set the way I explained and I only have my IOT VLAN set out send traffic out the gateway. 

Yes I did.

Here are my firewall rules: 
https://imgur.com/a/f6FMl7g.   I want to have my IOT VLAN the only one using it for devices off that LAN VLAN on those ports. 

Here are my NAT rules:
https://imgur.com/a/HVKfsQs

I'm trying to setup my OPNSense box as a VPN client to a commercial VPN provider.  I have installed the config file from the provider and have the VPN connection up.  I have added manual outbound NAT rules specifying the specific ports I want to go through the VPN tunnel.  When I initiate a connection utilizing those destination port I've specified to go through the tunnel (using an alias) the connection works but it doesn't go through the VPN tunnel.

Any suggestions I'm sure it's something silly I'm missing. 

Issue appears to be resolved.  For some reason the upstream Fiber shelf was caching the MAC address of the old firewall and not allowing ARP for the CARP address.  I'm up and running thanks for all your help folks!

I have added rules that allows traffic from each VLAN to anywhere yes. 

I've been playing with this a bit more.  I can't see what's going on.  I'm my Outbound NAT Manual Rules, I have the following example rules setup:

Interface:  WAN
Source:  ManagementVLAN net (my first VLAN), just for testing purposes.
Source/Destination/Destination Port: *
NAT Address (My WAN VIP for CARP)
Nat Port: *
Static Port NO

I can ping and resolve DNS just fine from the firewall itself under diagnostics, so traffic coming directly off the box is working fine.  As I haven't fully built the 2nd firewall yet, the VIP's all come up as master on this box. 

When I look at pftop I see traffic trying to come off devices on my LAN and go to external addresses out on the Net, but the state is Single: NO_TRAFFIC or NOTRAFFIC:SINGLE. 

I have 4 VLANS setup on the internal side and I can talk between them without issues from a machine on the management VLAN. 

I have recently started looking at OPNSense to replace my Untangle firewall after their recently announced licensing changes.  I have LAN connectivity just fine and I can connect out to the Internet from the firewall without issue (CLI I can ping out and resolve DNS).  Any traffic from one of the 4 VLANS I have makes it to the firewall but dies. I am intending to do a CARP/HA setup so I have that configured with the proper VIP's and the proper Manual outbound nat rule to nat all traffic coming from my internal networks (I setup a group for them) and I can't get any traffic through.  When I traceroute on a machine on one of the VLANs I get the first hop as the dedicated interface IP on the VLAN and then it dies from there. 

Any suggestions on where I could start looking?  I can provide any additonal information if needed to assist in pinpointing the issue, I've just recently started playing with OPNSense so I'm sure it's probably something I have set incorrectly.

Vihonator,

I agree with your assesment.  I currently work for the ISP providing me the service and they have a speedtest server on the same network about 2ms away from me.  With the CCR1009 I only get about 450 - 500Mbps on a speedtest due to the way the Tilera architecture works.  My ONT is connected right now to my data rack with a 7' Cat6 cable and comes up at a full gig.  My lan is where I have the 10G to my switch and it's a 3' fiber jumper directly from the Juniper into the Mikrotik currently.  I know I won't see the 1GBPS on my wifi network (which is spread across 5 Ruckus AP's in my house), I'm wanting the wire speed primarily on my main PC which is cat 6 right back to the Juiper over Cat6 and about a 20' run.  VPN I'm aware of, I don't need wire speed there. 

Hello,

I'm currently utilizing a Mikrotik CCR1009 and on small packets I only get about 450 - 500Mbps throughput, which hashes up with the specs they provide.  I'm getting ready to upgrade from cable to FTTH on a 1Gbps/500Mbps plan.  I'm looking to make the move to OPNSense.  I currently utilize a VPN (L2TP w/IPSEC) to send some traffic through (my MKT acts as a client to connect to the VPN Provider).  If I make the jump to OPNSense I'll likely switch to Wireguard.  Here is my desired functionality and a bit about my current setup:

LAN -- 10Gbps connection to my aggregation switch (Juniper EX4200)
VLANS -- 4 VLANS currently, I could see 6 VLANS at most.
Some Policy Based routing
Firewall

To do this with OPNSense, what type of hardware on the processor should I be looking at?  I want to be able to bounc at least 4 - 5Gbps through the LAN side and see full wire speed throughput on small packets from LAN to WAN out through the FTTH connection.  I would like to run IPS/IDS as well, which is something I can't really do with my Mikrotik currently.  The only other thing I'd be adding is a Wireguard Road Warrior setup for when I'm on the road.  I plan to get 2 of the same machines and use CARP for redundancy between them.  Ideally I'd like to spend no more than about 500 - 600 US on the hardware.

I was thinking/Looking at something like this (with an add-on PCI-E 10Gbps SFP+ card)
https://www.ebay.com/itm/SuperMicro-1U-customizable-Server-W-X9SCI-LN4F-E3-1270-V1-V2-8GB-32GB-DDR3/383421314359

I'd plan on 16GB of RAM and the E31270-V2. 

Would this hardware meet my needs, or is it too underpowered?

TYIA!