SOLVED: upgrade from 20.1 to 20.7 configuratiopn parse problem (probably)

[terxw]

Hi,

after running update from 20.1 to 20.7 and after that 20.7.1 firewall started behaving differently.

1. DNS stopped working
2. strange logs from unrelated filter rules were assigned to other ip ranges
3. ping ips e.g. 8.8.8.8 from lan stopped working event when firewall iteself was connected to internet, with working dns
4/ firewall log stopped updating right after reboot
 - after digging through logs I found corrupted filter.log, after last timestamp which corelated with last line in web gui, thera was binary data in txt log file

After dissecting backup xml and manual parcial application of working parts I found offending section, which was filter rules.
Even after manualy adding basic vlan rules, the firewall again stopped updating in web gui

Running in Qotom appliance e.g. x64 pc

working and not working diff:

Code: [Select]

--- /conf/backup/config-1598529328.7681.xml 2020-08-27 13:55:28.768757000 +0200
+++ /conf/config.xml 2020-08-27 18:03:17.561776000 +0200
@@ -1205,162 +1205,6 @@
         <any/>
       </destination>
     </rule>
-    <rule>
-      <type>pass</type>
-      <interface>lan</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>vlan 90 KaresWIfi to lan allow</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <network>opt2ip</network>
-      </source>
-      <destination>
-        <network>lan</network>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528372.929</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.111</username>
-        <time>1598528372.929</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>
-      <type>pass</type>
-      <interface>lan</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>vlan 80 Kares Guest o lan allow</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <network>opt3ip</network>
-      </source>
-      <destination>
-        <network>lan</network>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528401.006</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.111</username>
-        <time>1598528401.006</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>
-      <type>pass</type>
-      <interface>opt2</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>to vlan 90 allowed by list</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <address>Allow_vlan_net</address>
-      </source>
-      <destination>
-        <any>1</any>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528637.0106</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.222</username>
-        <time>1598477577.3571</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>
-      <type>pass</type>
-      <interface>opt2</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>lan to  vlan 90 allowed all</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <network>lan</network>
-      </source>
-      <destination>
-        <any>1</any>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528624.247</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.111</username>
-        <time>1598528549.786</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>
-      <type>pass</type>
-      <interface>opt3</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>to vlan 80 allowed by list</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <address>Allow_vlan_net</address>
-      </source>
-      <destination>
-        <any>1</any>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528652.0203</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.111</username>
-        <time>1598528652.0203</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>
-      <type>pass</type>
-      <interface>opt3</interface>
-      <ipprotocol>inet</ipprotocol>
-      <statetype>keep state</statetype>
-      <descr>lan to  vlan 80 allowed all</descr>
-      <direction>in</direction>
-      <log>1</log>
-      <quick>1</quick>
-      <source>
-        <network>lan</network>
-      </source>
-      <destination>
-        <any>1</any>
-      </destination>
-      <updated>
-        <username>root@192.168.50.111</username>
-        <time>1598528609.4877</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </updated>
-      <created>
-        <username>root@192.168.50.111</username>
-        <time>1598528609.4877</time>
-        <description>/firewall_rules_edit.php made changes</description>
-      </created>
-    </rule>
   </filter>
   <rrd>
     <enable/>
@@ -1416,9 +1260,9 @@
     <column_count>2</column_count>
   </widgets>
   <revision>
-    <username>root@192.168.50.111</username>
-    <time>1598528782.2619</time>
-    <description>Gateways: removed gateway 0</description>
+    <username>(system)</username>
+    <time>1598472328.7505</time>
+    <description>/usr/local/opnsense/mvc/script/run_migrations.php made changes</description>
   </revision>
   <OPNsense>
     <monit version="1.0.8">
@@ -2559,8 +2403,8 @@
       </client>
     </wireguard>
     <Interfaces>
-      <vxlans/>
-      <loopbacks/>
+      <vxlans version="1.0.1"/>
+      <loopbacks version="1.0.0"/>
     </Interfaces>
     <MDNSRepeater version="1.0.0">
       <enabled>0</enabled>
@@ -2721,5 +2565,5 @@
       </timerange>
     </schedule>
   </schedules>
-  <staticroutes/>
+  <staticroutes version="1.0.0"/>
 </opnsense>

Edit1: missing firewall log was caused by not starting/stopped syslog, see here:https://forum.opnsense.org/index.php?topic=18587.0
solution: upgrade to 20.7.2 or

Code: [Select]

pkg add -f https://pkg.opnsense.org/FreeBSD:12:amd64/20.7/misc/syslog-ng327-3.27.1_2.txz

Edit2: dns problems were caused by wrong IPV4 upstream GW for LAN,  in old config I had LAN_GW for lan, which was not working, changing the setting to auto-detect  corrected the blocking off all TCP