New user - Migrating from "pfSense - CE 2.7.0" to OPNsense

[Mega32]

Comming "From the Other side", and finally having had enough of their 180 degrees turns.
Looking forward to use a firewall where one hopefully can trust the promises that has been given."

I'we used "The Other one" for 5+ years, and am working w. enterprise networking daily.
The tech stuff doesn't scare me, but a totally new GUI layout ... It's gonna be steep.

Just installed the latest OPNsense on a Protecli i3 w. 6 IF's.
And am looking around in the gui right now.

Any hints on how2 migrate (import) bits of the "Other" config into OPNsense would be much appreciated.
Especially my IP/Network & Port aliases would be nice to "import"

NB: The below steps are only a suggestion/help in how to migrate a pfSense CE-2.7.0 config into OPNsense
There is absolutely no guarantee, that it will import everything correctly
Always check your migrated OPNsense config manually, and verify your setup, before setting the firewall into production.


Initial preparation of the pfS config file, before importing/restoring sections into OPNsense.
https://forum.opnsense.org/index.php?topic=36683.msg179265#msg179265

First hurdle: Import Aliases
https://forum.opnsense.org/index.php?topic=36683.msg179305#msg179305

Second hurdle: Import Vlans
https://forum.opnsense.org/index.php?topic=36683.msg179400#msg179400

Third hurdle: Import Certs and OpenVPN
https://forum.opnsense.org/index.php?topic=36683.msg179789#msg179789

Fourth hurdle: Import Interfaces
https://forum.opnsense.org/index.php?topic=36683.msg179463#msg179463

Fifth hurdle: Import Nat/Portforwarding
https://forum.opnsense.org/index.php?topic=36683.msg180141#msg180141

Sixth hurdle: Import firewall rules
https://forum.opnsense.org/index.php?topic=36683.msg180003#msg180003

ToDo ...
Move to a Physcical Box (Right now on VBOX)  - Moved to Protecli i3 6port - OK
Nat/Portforward  - Imported , untested yet
DNS
DHCP
Logging (Rules and Remote syslog)
Packages

[Seimus]

I think doing an export of PFsense config and than importing it into OPN is not a good option. In order to all the features/things you have its better to configure the new FW from scratch.

I would advise to scope what you have configured on PF and than reconfigure it on OPN. If you dont have a scope done you can do it manualy or use a script for it.

https://github.com/TKCERT/pfFocus

If you have only Interfaces, Rules, Aliases this can be fairly easy and fast.
https://forum.opnsense.org/index.php?topic=21083.15
https://forum.opnsense.org/index.php?topic=28209.0

* backup the config. It is XML so fairly readable
* Make screenshots of changed settings you remember
* Create a pfsense VM restore the mentioned backup so you can easily compare

From my experience OPN is not hard for configuration.

Regards,
S.

[franco]

Hi and welcome,

Unfortunately alias support was rewritten in 2019 which doesn't use the old storage location in the config.xml anymore. I think it would be possible to attempt a migration, but it's a bit tricky and the result may not be as desired.

Consider this just pointers to how this could work. No idea if layout changed too much between 2019 and now on their end. I can't vouch for this and it's not a goal we have as a project to be driven by external changes just to make this work.

Make sure you are on 23.7.7 first, then delete the new structure from the config.xml (it will kill all the aliases currently set up):

# pluginctl -f OPNsense.Firewall.Alias

(but a backup will be created)

Now edit /conf/config.xml to place your old aliases from the pfSense config, I think these were under <aliases/> tag directly below the root node "<pfsense/>".

When done run the migration and see what happens:

# /usr/local/opnsense/mvc/script/run_migrations.php

If not revert to the initial backup for safety.


Cheers,
Franco

[chemlud]

Hi there!

If you have a lab setup you could try to import bits and pieces, I did it in 2015, when both senses were pretty close. Maybe aliases and FW rules still working? Interfaces, NAT? Or you copy and paste with a text editor and import the .xml afterwards.

The design of the GUI is imho much more intuitive in OPN, but for sure you have to adapt. What I still miss are the separators for FW rules to mark groups of rules, but no way to get that feature out of the devs... ;-) trying for about 6-8 years now.

Enjoy the latest and greatest in OPN, as pfsense patching the CE once a year or so.

[Seimus]

Hi there!

If you have a lab setup you could try to import bits and pieces, I did it in 2015, when both senses were pretty close. Maybe aliases and FW rules still working? Interfaces, NAT? Or you copy and paste with a text editor and import the .xml afterwards.

The design of the GUI is imho much more intuitive in OPN, but for sure you have to adapt. What I still miss are the separators for FW rules to mark groups of rules, but no way to get that feature out of the devs... ;-) trying for about 6-8 years now.

Enjoy the latest and greatest in OPN, as pfsense patching the CE once a year or so.

Personaly In regards of the separators I use the Categories option, they allow to at least to search for the rules with same TAG. I know its not the same, but it came handy the moment I started to have like 5-10 VLANs.

Regards,
S.

[Monviech (Cedrik)]

Yeah but categories can be deceiving. Enabling one in Firewall and going to NAT for example and I was like multiple times "oh no everything is gone" xD, but the category was just still enabled.

[Mega32]

@all
Thanx for the current answers ... Keep'm comming

I have spend some time setting up a test OPNsense in VBOX.
It took some time as i had to "Bridge Wan" in order to be able to do https to it.
And i then had some issues if i used my Linux , where VBOX was installed.
As the browser (firefox) source to the VB OPNsense fwall....

I had to use another PC in order to be able to browse to the OPNsense (bridged ip) ...
Mega strange ... And i tried to allow "any .. any .. any" still no luck

I have allowed RFC1918 on WAN IF (settings).
I could connect if i did a : pfctl -d , but every time i changed something it "died" (prob enabling pfctl).
My guess would be that this is a VBOX issue ....


Well i'm now ready to play with a device that can do snapshots..

@Franco
I'll try that trick asap (but we have to go to dinner now)
I'll continue tomorrow

PS: I also donated a bit, in the sticky thread

[Mega32]

Initial Vlan/Vlan-Interface name adapting of the pfS "full config file".
Currently only needed if vlans are present in the pfSense config.

It turns out that pfSense and OPNsense doesn't use the same VLAN and Interface VLAN naming anymore.
And even though the OPNsense importer will import the current CE-2.7.0 Vlan names, the new pfSense naming might create other issues on the OPNsense.

Franco explains:

At a quick glance it's the new (incompatible) way the VLANs are named in CE-2.7.0:

<if>igb1.899</if>

The old compatible way would be igb1_vlan899, but that also requires
changing the VLAN device names as well.  This is going to be a manual
process. Otherwise the device will never be fully understood as a VLAN
and it could have more side effects during operation.

Before the pfS config file is being imported to OPNsense:
Vlan and interface names/referrals like the below:
igb1.100
must be changed to
igb1_vlan100


So we have to manually edit the pfS config file, and change some vlan and interface vlan names.

I made a manual search replace of these two combinations, as I only have vlans on em1 and em2
Watch out if you use "Replace All" ...

em1.
em1_vlan

em2.
em2_vlan

pfSense CE-2.7.0 naming of vlans.

Code: [Select]

<vlans>
<vlan>
<if>em1</if>
<tag>100</tag>
<pcp></pcp>
<descr><![CDATA[inside]]></descr>
<vlanif>em1.100</vlanif>
</vlan>
<vlan>
<if>em1</if>
<tag>110</tag>
<pcp></pcp>
<descr><![CDATA[new_inside]]></descr>
<vlanif>em1.110</vlanif>
</vlan>
<vlan>
<if>em2</if>
<tag>10</tag>
<pcp></pcp>
<descr><![CDATA[inet_only]]></descr>
<vlanif>em2.10</vlanif>
</vlan>

Adapted pfSense naming of vlans, before OPNsense import

Code: [Select]

<vlans>
<vlan>
<if>em1</if>
<tag>100</tag>
<pcp></pcp>
<descr><![CDATA[inside]]></descr>
<vlanif>em1_vlan100</vlanif>
</vlan>
<vlan>
<if>em1</if>
<tag>110</tag>
<pcp></pcp>
<descr><![CDATA[new_inside]]></descr>
<vlanif>em1_vlan110</vlanif>
</vlan>
<vlan>
<if>em2</if>
<tag>10</tag>
<pcp></pcp>
<descr><![CDATA[inet_only]]></descr>
<vlanif>em2_vlan10</vlanif>
</vlan>

pfSense CE-2.7.0 naming of vlan-interfaces.

Code: [Select]

<interfaces>
<lan>
<enable></enable>
<if>em1.110</if>
<descr><![CDATA[LAN]]></descr>
<spoofmac></spoofmac>
<ipaddr>192.168.110.1</ipaddr>
<subnet>24</subnet>
</lan>
<opt1>
<descr><![CDATA[inside_em1_VL100]]></descr>
<if>em1.100</if>
<spoofmac></spoofmac>
<enable></enable>
<ipaddr>192.168.17.1</ipaddr>
<subnet>24</subnet>
</opt1>
<opt2>
<descr><![CDATA[mgmt_em1_VL120]]></descr>
<if>em1.120</if>
<enable></enable>
<spoofmac></spoofmac>
<ipaddr>192.168.120.1</ipaddr>
<subnet>24</subnet>
</opt2>
<opt3>
<descr><![CDATA[inet_only_em2_VL10]]></descr>
<if>em2.10</if>
<spoofmac></spoofmac>
<enable></enable>
<ipaddr>192.168.11.1</ipaddr>
<subnet>24</subnet>
</opt3>


Adapted pfSense CE-2.7.0 naming of vlan-interfaces, before OPNsense import.

Code: [Select]

<interfaces>
<lan>
<enable></enable>
<if>em1_vlan110</if>
<descr><![CDATA[LAN]]></descr>
<spoofmac></spoofmac>
<ipaddr>192.168.110.1</ipaddr>
<subnet>24</subnet>
</lan>
<opt1>
<descr><![CDATA[inside_em1_VL100]]></descr>
<if>em1_vlan100</if>
<spoofmac></spoofmac>
<enable></enable>
<ipaddr>192.168.17.1</ipaddr>
<subnet>24</subnet>
</opt1>
<opt2>
<descr><![CDATA[mgmt_em1_VL120]]></descr>
<if>em1_vlan120</if>
<enable></enable>
<spoofmac></spoofmac>
<ipaddr>192.168.120.1</ipaddr>
<subnet>24</subnet>
</opt2>
<opt3>
<descr><![CDATA[inet_only_em2_VL10]]></descr>
<if>em2_vlan10</if>
<spoofmac></spoofmac>
<enable></enable>
<ipaddr>192.168.11.1</ipaddr>
<subnet>24</subnet>
</opt3>



Remember to do your own adaptations
Ie. I have connected my OPNsense WAN (dhcp) to my "Inside Vlan".
And i have changed the "Inside Vlan ip net" in my pfS config file, that's going to be used for OPNsense.
Else i would have same ip on the OPNsense Inside , as on the OPNsense Wan (connected to my pfS Inside Vlan).
The OPNsense Inside will get the original lan restored, as soon as it goes into prod.

[franco]

Hmm, this has to work one way or the other. Would you mind sharing sample config from your end so I can take a closer look? Best via mail to franco AT opnsense DOT org


Thanks,
Franco

[franco]

Apparently there was an issue in the migration that prevented nested aliases from validating during migration:

https://github.com/opnsense/core/commit/e4c857f0

So here is the full sequence on top of 23.7.7:

The patch will be included in 23.7.8 onwards so no need to run the patching there!

# opnsense-patch e4c857f0

(only apply the patch once otherwise it's going to be removed again)

# pluginctl -f OPNsense.Firewall.Alias
# pluginctl -f aliases

(confirm delete of garbage entries to allow for a migration)

edit /conf/config.xml to add your pFsense <aliases/> section cleanly again -- it should go right above the last line of the file saying </opnsense>

# /usr/local/opnsense/mvc/script/run_migrations.php
Migrated OPNsense\Firewall\Alias from 0.0.0 to 1.0.1

In case there are errors please post them here. They can be seen via the following command when the migration would fail.

# opnsense-log


Cheers,
Franco

[Mega32]

First hurdle: Import Aliases is done with the help from Franco 
Couldn't have done it without him 

New way to import aliases

Franco did some magic, and with a yet another patch, he made it possible to import the aliases section, directly from the pfS config file.

Apply the patch(es) (if you're on 23.7.xx) - Prob not needed if you're on next release:
NB: Only apply once, or the next apply will revert the patch.
opnsense-patch e4c857f0
opnsense-patch b8bbb00da7

Now yo should be able to import Aliases directly from the pfS config file.


Alias import done ...




The below steps are now obsolete:
I have made a shell script automating the import.

Requires a fully updated 23.7.7:
And a for now a patch - The patch might not be needed on later versions.

Apply the patch:
opnsense-patch e4c857f0

NB: The script will start out by erasing ALL existing aliases in the OPNsense config file  - /conf/config.xml
So only use it on a test machine with a "Clean config"

Once the aliases has been imported, you can export the aliases from the test machine as json or as a subset of the full config.
I would expect that file to import "Clean" on any other 23.7 machine.

See attached script, for instructions.
And as usual : Use at your own risk

[Mega32]

I just "took my own medicine" , and failed 

Code: [Select]

Do you want to flush this config property? [y/N]: y
Done. A backup was created and can be restored if needed.
Cannot find aliases   <---- This one can be ignored - It's due to the Alias section delete's performed by the script
*** OPNsense\Firewall\Alias Migration failed, check log for details  <---- Import Error
I forgot to apply the patch 

Code: [Select]

root@xxxx-fw-01:~ # opnsense-patch e4c857f0
Fetched 28df2b8fb via https://github.com/opnsense/core
Next run went fine , and my aliases appeared

[franco]

Can you change to patch e4c857f0 in your steps... I found a slightly related bug and that's the actual backport for 23.7.8.


Cheers,
Franco

[Mega32]

Sure
I just applied it, (wo. removing the other patch)  , it applied cleanly
And i corrected the patch# in the steps above.

Thnx

[Mega32]

I tried the "same patch trick" , on a pfS exported Vlan-section (attached just before the </opnsense> line) .
It didn't barf or output anything.
But no vlans appeared.

Update it does work ... If you remove the "empty" OpnSense "Vlan section" from config.xml

@Franco ... Any trick to do that ??

I mean like this one for Aliases

Code: [Select]

pluginctl -f OPNsense.Firewall.Alias