How to update MAC vendor db?

[Werner Fischer]

The status site "status_interfaces.php" shows the vendor of a MAC address.

We have now acquired the MAC vendor range DC58BC, which is now listed at IEEE, see
https://regauth.standards.ieee.org/standards-ra-web/pub/view.html#registries and http://standards-oui.ieee.org/oui/oui.txt

My question is: where does OPNsense store this oui.txt? (I'd like to contribute a patch, so that future OPNsense version show the vendor "Thomas-Krenn.AG" for MAC addresses starting with DC:58:BC  ;)

Thx and best regards,
Werner

[franco]

Hi Werner,

The database is pulled in via https://pypi.org/project/netaddr/

The library itself wasn't renewed since January 2017 though...


Cheers,
Franco

[xpac]

This is the most easy to find posting on this issue via Google.
As this is still an issue and the file is still as old as before, I've opened an issue on this:
https://github.com/opnsense/core/issues/3883

[XeroX]

Finally the Package including Databases have been updated (last more than 3 years ago)

Can we get an update of this package to have finally granular vendor mac addr lookup?

https://netaddr.readthedocs.io/en/latest/changes.html
Release: 0.7.20
Date: 19 Jun 2020

(This is at least the used library for ARP Table?!)
https://github.com/opnsense/core/blob/master/src/opnsense/scripts/interfaces/list_macdb.py

[XeroX]

Works like a charm on ARP Table. However breaks Vendor Lookup in DHCP completely.

distinfo

Code Select Expand


TIMESTAMP = 1600381665
SHA256 (netaddr-0.7.20.tar.gz) = d09252e5aec3913815d77eb8e8ea8fa6eb33521253e52f977f6abaa964776f3e
SIZE (netaddr-0.7.20.tar.gz) = 1889698


Code Select Expand

rm files/patch-setup.py

Makefile

Code Select Expand

--- Makefile.old        2020-09-18 00:29:47.359512000 +0200
+++ Makefile    2020-09-18 00:29:53.599067000 +0200
@@ -2,7 +2,7 @@
# $FreeBSD$

PORTNAME=      netaddr
-PORTVERSION=   0.7.19
+PORTVERSION=   0.7.20
PORTREVISION=  1
CATEGORIES=    net python
MASTER_SITES=  CHEESESHOP

EDIT:
DHCP Leases uses the following command:
configctl interface list macdb json
Execute error

Its broken as a method does not exist anymore:

Traceback (most recent call last):
  File "/usr/local/opnsense/scripts/interfaces/list_macdb.py", line 39, in <module>
    if os.path.isfile(netaddr.eui.ieee.OUI_REGISTRY_PATH):
AttributeError: module 'netaddr.eui.ieee' has no attribute 'OUI_REGISTRY_PATH'

https://github.com/netaddr/netaddr/commit/5b2807ff91640c959dc358334f626cea0e28778c

EDIT2:
0.8.0 released 6 days ago.

[XeroX]

Bad code and ARP Vendor Lookup works different than DHCP Leases. DHCP leases always parses the complete list, instead of looking up each single item with netaddr method. I'm not a programmer, I may check how to build this better.

Fix for DHCP:
/usr/local/opnsense/scripts/interfaces/list_macdb.py

Code Select Expand

--- list_macdb.py.old   2020-09-18 01:28:10.300640000 +0200
+++ list_macdb.py       2020-09-18 01:27:01.729241000 +0200
@@ -27,12 +27,11 @@
import os.path
import sys
import ujson
-import netaddr.eui.ieee

if __name__ == '__main__':
     result=dict()
-    if os.path.isfile(netaddr.eui.ieee.OUI_REGISTRY_PATH):
-        for line in open(netaddr.eui.ieee.OUI_REGISTRY_PATH, 'rb'):
+    if os.path.isfile('/usr/local/lib/python3.7/site-packages/netaddr/eui/oui.txt'):
+        for line in open('/usr/local/lib/python3.7/site-packages/netaddr/eui/oui.txt', 'rb'):
             line = line.decode()
             if line.find('(base 16)') > -1:
                 parts=line.split('(base 16)')


[franco]

Is that not exactly the same? Besides, hardcoding 3.7 will break on the next Python update.


Cheers,
Franco

[XeroX]

Morning franco,
what do you mean exactly the same? referred to?

I know it breaks, will try to improve that, I'm quiet new to python.

[franco]

Hi there :)

This looks for a file:

if os.path.isfile(netaddr.eui.ieee.OUI_REGISTRY_PATH):

This looks for a file:

if os.path.isfile('/usr/local/lib/python3.7/site-packages/netaddr/eui/oui.txt'):

It is probable that netaddr.eui.ieee.OUI_REGISTRY_PATH actually is /usr/local/lib/python3.7/site-packages/netaddr/eui/oui.txt ;)


Cheers,
Franco

[XeroX]

Oh yes, franco. You're correct.

Check the commit I posted on netaddr. The method has been removed. That's why changed it.

https://github.com/netaddr/netaddr/commit/5b2807ff91640c959dc358334f626cea0e28778c

EDIT:
Here we go:

Code Select Expand

--- list_macdb.py.old   2020-09-18 10:42:50.178232000 +0200
+++ list_macdb.py       2020-09-18 10:39:20.059022000 +0200
@@ -27,12 +27,12 @@
import os.path
import sys
import ujson
-import netaddr.eui.ieee
+import imp

if __name__ == '__main__':
     result=dict()
-    if os.path.isfile(netaddr.eui.ieee.OUI_REGISTRY_PATH):
-        for line in open(netaddr.eui.ieee.OUI_REGISTRY_PATH, 'rb'):
+    if os.path.isfile((imp.find_module('netaddr')[1])+'/eui/oui.txt'):
+        for line in open((imp.find_module('netaddr')[1])+'/eui/oui.txt', 'rb'):
             line = line.decode()
             if line.find('(base 16)') > -1:
                 parts=line.split('(base 16)')


[franco]

Thanks, this was added to the development branch. We will work on a netaddr update right after 20.7.3 is out.


Cheers,
Franco

[XeroX]

Great franco, thank you!

Hope you can implement 0.8.0 directly and skip 0.7.20.

If you need any help or pull request let me know.

Which branch is it?

[franco]

More discussion here. I'm not sure where this stands in FreeBSD as there is some kerfuffle about Python version support.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249382


Cheers,
Franco

[XeroX]

Any updates on this?

[franco]

I asked on the FreeBSD bug tracker. Another ticket here https://github.com/opnsense/ports/issues/110