[SOLVED] parse failure in list_arp.py causes empty Intf/Diag/ARP list

[greg124816]

Just registered to report this error I ran into after to migrating to OPNsense.

I am pretty sure I have the issue pinned down to the lease.find and dhcp_ipv4_address assignment in:

Code: [Select]

leases = open(dhcp_leases_filename, 'r').read()
        for lease in leases.split('}'):
            if lease.find('{') > -1:
                dhcp_ipv4_address = lease.split('{')[0].split('lease')[1].strip(
)
                if lease.find('client-hostname') > -1:
                    dhcp_leases[dhcp_ipv4_address] = {'hostname': lease.split('client-hostname')[1].strip()[1:-2]}
Things work good when only "lease" objects appear in dhcpd.leases file. They still work when the "failover peer" object appears as the very first object, just after the ISC boilerplate comments at the top.

When the "failover peer" object appears further down after other "lease" object(s), like this:

Code: [Select]

lease 192.168.99.242 {
  starts 2 2016/12/13 18:05:23;
  tstp 2 2016/12/13 18:05:23;
  tsfp 2 2016/12/13 18:05:23;
  atsfp 2 2016/12/13 18:05:23;
  binding state backup;
}
failover peer "dhcp_lan" state {
  my state normal at 2 2016/12/13 16:30:14;
  partner state normal at 2 2016/12/13 17:05:23;
}
I'm guessing the lease.find("lease") returns an array with a single item... at index 0, which then causes the error:

Code: [Select]

root@fwa:/usr/local/opnsense/scripts/interfaces # ./list_arp.py
Traceback (most recent call last):
  File "./list_arp.py", line 51, in <module>
    dhcp_ipv4_address = lease.split('{')[0].split('lease')[1].strip()
IndexError: list index out of range
root@fwa:/usr/local/opnsense/scripts/interfaces #

The reason it works normally is that the failover peer object appears first after the ISC comments as shown here:

Code: [Select]

# The format of this file is documented in the dhcpd.leases(5) manual page.
# This lease file was written by isc-dhcp-4.3.5

# authoring-byte-order entry is generated, DO NOT DELETE
authoring-byte-order little-endian;


failover peer "dhcp_lan" state {
  my state normal at 2 2016/12/13 16:30:14;
  partner state normal at 2 2016/12/13 17:05:23;
}
The comments above the lease object contain the token "lease" (twice!), so that lease.split[0] and [1] exist.

So the leases file gets parsed to a non ip address with blank hostname for one item in a failover setup. But there is no corruption of the arp table output by the script since lease.split[1] contains text and not an ip address, plus no client-hostname is found for this iteration and never gets used to provide hostname detail to any actual arp -a table IP that are looked up later in list_arp.py. At least that is my assumption, I didn't read any further down.

I'm not sure why my failover peer object appeared further down in the leases file. I just know that it's written at the discretion of dhcpd. While I was troubleshooting things the second failover peer object was cleared by the running DHCP server and only appeared at the top again... and my WebGui ARP Table output was working.

I don't know Python much but it seems if you used whatever needed to perform the following:

Code: [Select]

if (lease.find('{') > -1) && (lease.find(something that matches regex "^lease") > -1):
it might fix the problem.

Thanks!

[greg124816]

Replying to myself here, I realized I didn't' put in the actual error that is seen in the WebGui System/Log(after loading an empty Intf/Diag/Arp list):

Code: [Select]

configd.py: [bdfbd41d-060f-46de-969c-c67b1adc0dc0] Script action failed with Command '/usr/local/opnsense/scripts/interfaces/list_arp.py "json"' returned non-zero exit status 1 at Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 477, in execute stdout=output_stream, stderr=error_stream) File "/usr/local/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '/usr/local/opnsense/scripts/interfaces/list_arp.py "json"' returned non-zero exit status 1
The error I showed in my previous post was from ssh'ing in and running the script from the command line.

[franco]

Hi there,

Thanks, I will forward this to Ad to review/include.


Cheers,
Franco

[franco]

Ad committed the following:

https://github.com/opnsense/core/commit/feb11e59

On a non-modified version the patch applies by running the following command:

# opnsense-patch feb11e59

Let us know if that fixes the problem. It's a candidate for 16.7.x.


Cheers,
Franco

[greg124816]

Sorry for ignoring this for so long. At the time I was trying to move a pair of firewalls to a new hardware in a new location. I had just learned about the whole opnsense story and was trying to migrate to it at the same time.

I ended up only having time to restore my backed up configuration to a new install of my old firewall software.

This morning I finally had time to complete the migration. I had built up most of my configurations on two VM installs of opnsense so I could more quickly migrate.

I don't have my (new again) failover hardware running yet but I should have it in place before Monday and will be looking at the DHCP leases report with failover peer configured.

Thanks for implementing the fix!

greg

[franco]

Hi Greg,

Sure, thanks for checking back.


Cheers,
Franco

[greg124816]

Confirmed working.
I made sure both of the leases files had multiple peer entries in them throughout:

Code: [Select]

root@fwa:/var/dhcpd/var/db # grep -n peer dhcpd.leases*
dhcpd.leases:8:failover peer "dhcp_lan" state {
dhcpd.leases:323:failover peer "dhcp_lan" state {
dhcpd.leases:328:failover peer "dhcp_lan" state {
dhcpd.leases:333:failover peer "dhcp_lan" state {
dhcpd.leases~:8:failover peer "dhcp_lan" state {
dhcpd.leases~:323:failover peer "dhcp_lan" state {
dhcpd.leases~:328:failover peer "dhcp_lan" state {
dhcpd.leases~:333:failover peer "dhcp_lan" state {
root@fwa:/var/dhcpd/var/db #

Interfaces/Arp page displays perfectly fine.

Thanks again for fixing and sorry for the delay on my end.
Greg