Messages

Oh, well that sounds like a cue for some optimization!... I wonder what are the technical bottlenecks?

Oh damn, FUN: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856

Update here, rather than in the original post: the culprit seems to be the pppoe interface, as when I 'double nat' opnsense through the provided sagem router, it then taking care of the pppoe connection, I get a respectable 920/830!

What should I be investigating other than the two arbitrary tweaks on pppoe & vlan I mentioned above?

I've done basic due diligence in searching for posts with similar setup, and at this point will add mine:

I just got setup with symmetric 1G FTTH service and was consistently hitting 960/960Mbps with the provided consumer grade Sagem router, connected to a Nokia ONT.

Having acquired what I thought was a decent low powered machine I got to ditching the Sagem:
- supermicro X11SBA-LN4F, https://www.supermicro.com/en/products/motherboard/X11SBA-LN4F
- Quad Intel i210-AT nics igb(4), max of 8G of Crucial RAM, Kingston 240GB UV500 SSD mSATA

The opnSense install went without a hitch, save for insisting kern.vty=sc to not have the vga hang (something to do with the embedded graphics, don't fully understand this <shrug>), and rebooting to complete the 20.7.4 update.

As per our local ISP (Bell Canada) config I have VLAN 35 created on that igb0, then pppoe on top of that as the WAN. The WAN iface goes up... but the speedtests.net leave something to be desired, reporting consistently: ~320/440Mbps (interesting that UP is faster).

So far I have started tuning the following, according to some docs & random hearsay:
- kern.ipc.nmbclusters=1000000   
- kern.ipc.nmbjumbop= 1000000 (interstingly this helped to bring UP to 500Mbps)
- hw.igb.fc_setting=0
- set VLAN priority (pcp) to 6... stab in the dark (?)
- set the pppoe MTU size to 1600 from default 1492... again, stab in the dark

I admit, perhaps I had expected too much out of this 10W setup, but like to think it can do better!

Thanks for any suggestions, Mike

Yes, agreed, probably be able to run DHCP on a vnet, but again this would not be practical IMO. Yes, in a bind I can do static ip configuration assuming I get the network right, or insert contrived UDRs; but long term may run into ip address contention as someone else deploys a vm provisioned by default with DHCP.

For now, simply provisioning with WAN as my first Azure NIC does the trick.

Hazmat suit on ;), working on Azure,
Mike

deZillium, I'm with you, order of the iface should not matter, but this is Azure we're talking about ;) And the culprit in this case is the azure vm agent preinstalled on the freebsd image. Among all the magical things it does, it manages the route table on the host, and having picked up the Azure NIC 1 / hn0, it had decided that was my default gw. For all the magical things it does, good and bad*, for the moment I will live with it rather than gutting it out.

I managed my setup by swapping the virtual NICs across the subnets, and with hn0 now assigned to WAN everything worked as expected.

My networking-foo level is intermediate, but I'm not sure if that's a hindrance or an advantage when working with unconventional networking environments like Azure SDN. As far as I can tell, DHCP is the only practical way for ip configuration in an Azure vnet, as this is how Azure fabric knows how to route the traffic. (Unpractical ways involve using user defined routes to tell azure how to reach the host that happens to be listening to 192.168.1.1 by default; yes I managed this ;) ).

Yes again I'm with you, in a conventional networking sense I am running a router, and "of course" should be responsible for running DHCP on my LAN. But all the Azure documentation points to the fact that this is not supported, e.g. some of many:
- https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-faq#what-protocols-can-i-use-within-vnets
- https://support.microsoft.com/en-us/help/2721672/microsoft-server-software-support-for-microsoft-azure-virtual-machines (read down to section on "The following roles are not supported on Microsoft Azure virtual machines:"

deZillium, this is not to rebuff your knowledgeable suggestions, THANK YOU, but more to vent my Azure growing-pains. And yes, as per your suggestion, next time I am faced with this I will see what happens if I play with static ip assignment on the host (opnsense config), using the Azure fabric assigned ip.

Cheers, Mike

* The vm agent at this development stage is useful for basic host monitoring, password recovery, and my favorite I'm not willing to give up just yet: the Azure VM Serial console.

[*** I believe I could simply reassign the NICs effectively swapping to WAN -> hn0, and LAN -> hn1, but I want to better understand the OPNsense system, and the correct way of setting this up regarless of iface enumeration.]

Going to summarize the OPNsense Azure deployment, and ask a trivial question.

I've bootstrapped 19.1.0 (updated to 19.1.4) on Azure using Microsoft built FreeBSD 11.2, with two intefaces:
- LAN hn0 interface connected via DHCP to 'default' subnet 10.0.0.0/16
- WAN hn1 interface connected via DHCP to 'ext' subnet 168.10.0.0/16

(Recall that on Azure you use Azure DHCP or the highway!)

Firstly, the order the NICs are assigned in the Azure VM and the order they're enumerated by OS is following some form of convention, ie LAN as the primary, WAN + OPT as the gateways out, ? With this ordering of intefaces, LAN hn0 DHCP having found a default gw at 10.0.0.1, OPNsense will accept that as the system default gw as show it accordingly in System>Routes>Status.

This is of course bogus, compliments of Azure pseudo-networking (SDN), as the systems default gw should be that fetched by the WAN hn1 interface.

I've tried to find ways to override gateway settings in the Interfaces>[LAN]/[WAN], and also in the System>Gateways>Single pane where I've disabled LAN_DHCP as gateway. I do understand that the Gateways that appear in the latter are part of the configuration of OPNsense services, but I'm just stabbing in the dark in having WAN hn1 recognized as the default system gw.

*** I believe I could simply reassign the NICs effectively swapping to WAN -> hn0, and LAN -> hn1, but I want to better understand the OPNsense system, and the correct way of setting this up regarless of iface enumeration.

Again, this is bogus fundamentally, and then affects basic services like Outbound NAT where LAN is used as the default Outbound NAT interface. I've made some progress in wrangling a working NAT setup on this Azure vnet, #1 with the required Azure User Defined Routes (UDRs), and then setting the Outbound NAT manually... but there must be a correct way about this.

QUESTION: What is the correct way of setting the system default gw?

Look forward to being schooled in basic OPNsense 'sense', and swapping insights re cloud deployments.

Mike

Hi folks, I kicked the 'ol consumer router to the curb, and finally setup a proper inet frontend with opnsense. This is new to me but I'm keen to learn, so am left a bit puzzled when I'm seeing the following firewall logs claiming to be blocking traffic.

My setup is as follows, centos 7.4 KVM host: WAN (macvtap) connected to the modem, LAN (macvtap) connected to switch, and additionally an OPT1 for KVM host connectivity. Switch then has a wifi router (Google/TPLink On Hub) subnetting to wifi hosts & some other media appliances.

opnsense LAN is dishing out DHCP over 192.168.42.1/24, and the On Hub does it's thing in a 192.168.86.1 subnet.

... so everything is working great! Except as I explore firewall logs I'm seeing a slew of blocks into LAN (vtnet1)... traffic that seems to be getting NAT'ed just fine (client connectivty etc):

Code Select Expand


May 6 00:53:19 filterlog: 9,,,0,vtnet1,match,block,in,4,0x0,,63,2082,0,DF,6,tcp,40,192.168.86.22,52.206.150.146,40048,9543,0,A,,3932977683,2730,,
May 6 00:52:07 filterlog: 9,,,0,vtnet1,match,block,in,4,0x0,,63,60382,0,DF,6,tcp,87,192.168.42.2,108.177.14.188,46003,5228,35,PA,775150375:775150410,610185908,1546,,nop;nop;TS


The above are clients from both opnsense & On Hub subnets, and the dest ip are most often google & amazon hosts, that when plugged into browser return the google/amazon landing pages just fine.

So again, my opnsense seems to be working correctly, but I want to understand what these firewall logs are about. I feel as though I'm chasing a red herring, and suspect you'll school me in basic opnsense/freebsd foo.

Would appreciate it, Thanks,
Mike