Script to send emails for updates

[bartjsmit]

This script follows on from the example in the 'Using the API Howto' wiki entry. https://wiki.opnsense.org/index.php/Howto_use_the_API

Replace the redacted api_key and api_secret strings with your own and set the URL of your OPNsense instance as well as your name/email address.

It requires an MTA (set to localhost in the script) and can be run from cron. Tested with Python 2.7.5-34 on CentOS 7.

[franco]

Hi there,

This is really cool, thanks for sharing. I'm making this sticky in the general forum.


Cheers,
Franco

[philamonster]

Thanks for this. Confirmed on Debian wheezy. Got an update email today for 16.1.

[franco]

"Dude, where's my 16.1?"

[bartjsmit]

As some of you will have noticed, there is an issue with the script versus the Python request library, or rather with my (lack of) understanding of the latter.

Whereas before all JSON elements were returned as (unicode) list, some are now dict variables, causing the script to bomb out on a type error.

Attached is version 1.1 of fw_update.py, that tests for the returned type. As before, you can un-redact it and run it from cron.

Tested with Python 2.7.5-34 and python-requests-2.6.0-1 on CentOS 7

Enjoy!

Bart...

[philamonster]

Thanks for update. Working on Debian 7.9 w/python 2.7.3.

[TheLatestWire]

The script was working for me until I just updated python on my Gentoo system.  Now I get this when I run it.  Any ideas would be greatly appreciated.

(163) -->  python2.7 fw_update1.1.py
/usr/lib64/python2.7/site-packages/urllib3/connectionpool.py:790: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
Traceback (most recent call last):
  File "fw_update1.1.py", line 28, in <module>
    response = json.loads(r.text)
  File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

[bartjsmit]

The unverified HTTPS points to a x509 certificate chain issue, rather than Python.

Make sure you have a trusted certificate bound to the OPNsense webConfigurator under Certificates and that its root CA as well as any intermediate CA's in the trust chain are imported under Authorities. Both menus are under System:Trust in the web interface. Don't forget to restart the webConfigurator afterwards.

You can either create an internal CA for this and set your Gentoo to trust it,  or you can get a publicly trusted certificate for your firewall. I rent a cheap wildcard certificate from startssl.com for my home network.

If you just want stuff to work without worrying about certs, you can of course set verify=False in the request.get line, but Python will lecture you (quite rightly) about your poor OpSec 

Bart...

[TheLatestWire]

I have verify set to FALSE in the script -> "r = requests.get(url,verify=False"

It's the other errors that I don't know what to do with.

Traceback (most recent call last):
  File "fw_update1.1.py", line 28, in <module>
    response = json.loads(r.text)
  File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

I'm using Python 2.7.11 on Gentoo.

Thanks.

[TheLatestWire]

Upon further investigation I'm running python 2.7.11-r2 which is listed as unstable in the Gentoo dev-lang/python package.  Apparently it's nearly impossible to move back the stable branch of python due to reverse dependencies.

So maybe 2.7.11-r2 is the issue and it would work with 2.7.10-r1?

[TheLatestWire]

I'm using this script on another deployment of OPNsense in an environment where I only have an OpenBSD server to use for the SMTP service of this script and I'm not able to send the script's email using that server.  I get this error:

Traceback (most recent call last):
  File "fw_update-1.1.py", line 58, in <module>
    s.sendmail(sender,recipients,message)
  File "/usr/local/lib/python2.7/smtplib.py", line 746, in sendmail
    raise SMTPDataError(code, resp)
smtplib.SMTPDataError: (550, '5.7.1 Delivery not authorized, message refused: Message is not RFC 2822 compliant')

Is anyone able to use an OpenBSD mail server to deliver this script without this error by chance?

This is what it looks like from the OpenBSD smtp server side:

<<< [MSG] <br>Click <a href="https://192.168.2.1/ui/core/firmware/">here</a> to fetch them.<br>
debug: 0x18f9f6123000: end of message, msgflags=0x8000
smtp: 0x18f9f6123000: >>> 550 5.7.1 Delivery not authorized, message refused: Message is not RFC 2822 compliant
smtp-in: Failed command on session 48cc59eadf55d2f9: "data" => 550 5.7.1 Delivery not authorized, message refused: Message is not RFC 2822 compliant
smtp: 0x18f9f6123000: STATE_BODY -> STATE_HELO
smtp: 0x18f9f6123000: <<< rset
smtp: 0x18f9f6123000: >>> 250 2.0.0: Reset state
smtp-in: Received disconnect from session 48cc59eadf55d2f9
debug: smtp: 0x18f9f6123000: deleting session: disconnecte

Thanks.

[bartjsmit]

Most likely cause for a 5.7.1 error is that your FreeBSD MTA denies relay from your firewall.

Can you get a message to the recipient manually through a telnet session on port 25?

Bart...

[TheLatestWire]

I'm able to send a test email using this python script from the OPNsense server via the same OpenBSD mail server that is returning the "not RFC 2822 compliant" message for the fw_update-1.1.py script.

#!/usr/bin/python
import smtplib
sender = 'myname@mydomain'
receivers = ['myname@mydomain]
message = """From: From OPNsense <myname@mydomain>
To: To Person <myname@mydomain>
Subject: SMTP e-mail test
This is a test e-mail message.
"""
try:
   smtpObj = smtplib.SMTP('192.168.2.2')
   smtpObj.sendmail(sender, receivers, message)
   print "Successfully sent email"
   except SMTPException:
   print "Error: unable to send email"


Thanks.

[franco]

Well, what is the IMF output of the script? I guess the error is pretty clear, just need to figure out what is wrong with the message that should be dispatched.

[TheLatestWire]

Sorry for the stupid question, but what is IMF?  I'm just coming up with "International Monetary Fund" but I'm sure that's not what you mean.