FreeBSD Install

[jclendineng]

Hello, I am trying to install this on a dedicated box inside my opnsense network.  I know this runs on FreeBSD, but my question is, how do I get it to route inline? I do not see any documentation on this besides how to install (which works fine).  My initial thought was to bridge 2 ports, run sensei on that bridge port in normal routed mode, and have that inline.  So basically:

Modem ---> opnsense wan --> opnsense lan --> FreeBSD bridge with sensei (OR opnsense secondary server) --> internal network

Would this work? Just trying to understand how to get sensei up and running on a dedicated server (I dont want to do a full opnsense install on the second server as that seems unneeded and a waste of resources).  I could install a second opnsense inline but Im not sure that would work.

Thanks!

[mb]

Hi @jclendineng,

The best way to accomplish is to deploy sensei in bridge mode (which will be available with Sensei 1.10 scheduled for the end of July). In this mode Sensei establishes a virtual switch between two ethernet adapters - you'll need the third adapter for maintenance.

Till that time, let us try to get you some instructions for you to get this working manually.

Can you reach out to support through the Cloud Portal?

[jclendineng]

Thanks!  I actually did some testing and it seems to work the way I thought...for people on freebsd here is what I did:

Create the bridge and add nics, replace with your nics/ip:

ifconfig bridge create
ifconfig bridge0 addm igb0 addm igb1 up
ifconfig igb0 up
ifconfig igb1 up
ifconfig bridge0 inet 10.0.0.1/26
ifconfig igb0 mtu 9000
ifconfig igb1 mtu 9000
ifconfig bridge0 mtu 9000

Now edit the rc.conf file:

vi /etc/rc.conf

hostname="sensei"
ifconfig_bridge0="inet 10.0.0.1 netmask 255.255.255.192" # bridge information
ifconfig_igb0="up" # Make sure both bridged interfaces come up
ifconfig_igb1="up"
cloned_interfaces="bridge0"
autobridge_interfaces="bridge0"
autobridge_bridge0="igb0 igb1"
ifconfig_igb5="inet 10.0.2.52 netmask 255.255.255.192" # My management interface so I dont lock myself out
defaultrouter="10.0.2.1" # Default gateway

I then set up a cloud firewall and select "igb1" in my case and routed mode.  igb0 in my setup is connected to the lan out and igb1 is connected to the internal switch, so I think both nics will have the same information, only need to select 1.

I did run into another issue though, a pure cloud install does not let you resolve dns entries via the gateway, so all you see are ip addresses not hostnames.  I can confirm though that I am filtering just fine on freebsd with manual bridging.  Is DNS resolution coming in 1.10, and/or should I submit a bug report/feature request?


Edit: I removed all nameservers from the resolv.conf and added my main gateways (Im assuming it uses the OS nameservers to resolve IP's) and I also added a manual internal host to the host file, and then I lost access

The bridge is still up but I think I screwed up something, I have internet but looks like ill need to hook a monitor up as I lost ssh access.  The experiment continues! I was quite happy I got it mostly working though using a manual bridge

[mb]

@jclendineng, wow! congratulations. Are you on Passive Mode or L3 Mode? I'm surprised to see netmap is working with FreeBSD bridge(4) interface.

Reverse DNS resolution is coming very soon. It's on our plate along with MAC address based reporting / filtering.

[jclendineng]

L3 standard netmap mode.  I was surprised as well but it does work.  Blocking works as well as live traffic view along with full vlan information...I have 3 vlans that would be visible to that interface and all 3 are properly tagged so thats nice.  I would have liked to select only the bridge0 interface but I do not think thats possible since it uses netmap to directly talk to each interface and bridge0 is virtual, so would not be visible.  I of course need to re-install since I killed my ssh access but Ill wait for 1.10 most likely.  Its looking very promising.

[mb]

@jclendineng, thanks for the additional information.

I get it now. You've deployed sensei on the bridge member interfaces. Nice. That should work indeed since we're deploying on the member interface.

[jclendineng]

Hi @jclendineng,

The best way to accomplish is to deploy sensei in bridge mode (which will be available with Sensei 1.10 scheduled for the end of July). In this mode Sensei establishes a virtual switch between two ethernet adapters - you'll need the third adapter for maintenance.

Till that time, let us try to get you some instructions for you to get this working manually.

Can you reach out to support through the Cloud Portal?

Is this still scheduled for end of July? Very much looking forward to bridge mode on FreeBSD

[mb]

@jclendineng, we're still planning to include bridge mode in the next release (1.10). But it got delayed a bit. We had to re-architect the cloud agent; so that it can support a wider range of platforms.

Chances are high that we'll make it ready for the end of August.

[jclendineng]

Any updates on this?

[mb]

Hi @jclendineng, we'll test it this week. Still hopeful that it will make 1.10

[almodovaris]

Hi, I have OPNsense 23.1 and Zenarmor 1.12.4.

Zenarmor cannot create a stable bridge. But if I create the bridge manually, as shown above, it works okay with the bridge.

It runs on an APU2C4 or APU2E4 (I don't remember which of the two).

[mb]

Hi @almodovaris,

It's expected that if you use L2 Bridge Mode (netmap), whenever you stop the packet engine the bridge will go down - since bridge is managed by zenarmor itself.

Thanks for letting us know that you can protect a bridge member interface.

In the meantime, we're working with OPNsense Klara team to bring a better bridge(4) support for netmap:
 
https://reviews.freebsd.org/D38066

[almodovaris]

About "It's expected that if you use L2 Bridge Mode (netmap), whenever you stop the packet engine the bridge will go down - since bridge is managed by zenarmor itself.": Zenarmor tries to create a bridge or kind of, but I have no connectivity. With the manually created bridge I get connectivity through Zenarmor. If it matters, the bridge is created after Zenarmor starts.