Constant DNS queries for opnsense.emergingthreats.net

[nikon112]

After enabling Unbound DNS reporting I am seeing over 40,000 DNS queries for opnsense.emergingthreats.net over the course of six hours.

I am using Unbound (no blocklist) on opnsense with DoT to nextdns.
The queries all Pass and come back NOERROR with the vast majority being answered from Cache.
Since the queries are mostly being answered from cache they don't show up on nextdns, which is why I had not noticed before.
To be clear the queries are also not being blocked by nextdns.

Is anyone else seeing this issue, or know how to fix it?

Thanks.

[Fright]

using os-etpro-telemetry ids rules plugin?
"When you allow your OPNsense system to share anonymized information about detected threats - the alerts -
you are able to use the ETPro ruleset free of charge."

[nanoguy]

Getting the same, only the number of queries is much larger for me. Anyone got any idea how to mitigate against this?

[OpalALeslie]

After enabling Unbound DNS reporting I am seeing over 40,000 DNS queries for opnsense.emergingthreats.net over the course of six hours.

I am using Unbound (no blocklist) on opnsense with DoT to nextdns.
The queries all Pass and come back NOERROR with the vast majority being answered from Cache.
Since the queries are mostly being answered from cache they don't show up on nextdns, which is why I had not noticed before.
To be clear the queries are also not being blocked by nextdns.

Is anyone else seeing this issue, or know how to fix it?

Thanks.

Excessive DNS query issue for opnsense.emergingthreats.net within six hours when using Unbound DNS on OPNsense, while users search for solutions to minimize the continuously generated traffic.

poppy playtime chapter 3

[Patrick M. Hausen]

Don't use Suricata?

[someone]

Original question sounds like DNS or a misbehaving schedule possibly
Is your DNS sticking to its set IP
DNS settings in unbound
set your servers in system general
have the correct settings in unbound

[someone]

Are you behind a IPS router
Did you reset it before you went online
When you load opnsense, download your rules and apply them, and make your changes
I would create a snapshot and click it to be active, so thats what will be booted on the next powerup
After changes I would make another snapshot
Are you capturing packets or looking at them or the traffic
Once you set your DNS servers and reboot, look for your DNS server IP
Does your IPS let you select your own DNS or have to use theirs
Make sure let ISP over ride your settings is unchecked
Are you using firefox
In the settings under privacy and security
At the bottom check use your own DNS servers
Do you have a ET schedule activated
I am not getting that traffic
But I was getting DNS bombs, not any more

[someone]

check the box flush cache on reboot, then reboot, check logs

[agreenbhm]

I noticed today I was experiencing the same issue.  Thankfully I believe I've found a solution (and submitted a pull request to address it).  In short, the ET-Pro telemetry plugin is looping (every minute) through all collected events and submitting them to opnsense.emergingthreats.net.  The way the script is written though results in a separate DNS lookup and connection for every single event, which might be hundreds or thousands in even a smaller network.  The script in question is /usr/local/opnsense/scripts/etpro_telemetry/send_telemetry.py.  The simple solution was to add a requests.Session() object and use that for POSTs (resulting in a single DNS lookup) rather than spawning a new object for every request.  See below for the complete script that you can drop-in place if you'd like.

Code Select Expand

#!/usr/local/bin/python3

"""
    Copyright (c) 2018-2019 Ad Schellevis <ad@opnsense.org>
    All rights reserved.

    Redistribution and use in source and binary forms, with or without
    modification, are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice,
     this list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright
     notice, this list of conditions and the following disclaimer in the
     documentation and/or other materials provided with the distribution.

    THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
    INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
    AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
    AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
    SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
    INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
    CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
    POSSIBILITY OF SUCH DAMAGE.
"""

import sys
import os
import argparse
import requests
import time
import random
import syslog
import urllib3
import ujson
import telemetry.log
import telemetry.state

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

parser = argparse.ArgumentParser()
parser.add_argument('-e', '--endpoint', help='Endpoint url to reach',
                    default="%s/api/v1/event" % telemetry.BASE_URL)
parser.add_argument('-i', '--insecure', help='Insecure, skip certificate validation',
                    action="store_true", default=False)
parser.add_argument('-c', '--config', help='rule downloader configuration',
                    default="/usr/local/etc/suricata/rule-updater.config")
parser.add_argument('-l', '--log', help='log directory containing eve.json files',
                    default="/var/log/suricata/")
parser.add_argument('-s', '--state', help='persistent state (and lock) filename',
                    default="/usr/local/var/run/et_telemetry.state")
parser.add_argument('-d', '--days', help='Maximum number of days to look back', type=float, default=1)
parser.add_argument('-D', '--direct',
                    help='do not sleep before send (disable traffic spread)',
                    action="store_true",
                    default=False)
args = parser.parse_args()


exit_code = -1
send_start_time = time.time()
telemetry_state = telemetry.state.Telemetry(filename=args.state, init_last_days=args.days)
if not telemetry_state.is_running():
    cnf = telemetry.get_config(args.config)
    if cnf.token is not None:
        if os.path.isdir(args.log):
            last_update = telemetry_state.get_last_update()
            event_collector = telemetry.EventCollector()
            row_count = 0
            max_timestamp = None
            for record in telemetry.log.reader(args.log, last_update):
                if max_timestamp is None or record['__timestamp__'] > max_timestamp:
                    max_timestamp = record['__timestamp__']
                event_collector.push(record)
                row_count += 1
            # data collected, log and push
            if row_count > 0 and max_timestamp is not None:
                syslog.syslog(
                    syslog.LOG_DEBUG,
                    'telemetry data collected %d records in %.2f seconds @%s' % (
                        row_count, time.time() - send_start_time, max_timestamp
                    )
                )
                # spread traffic to remote host, usual cron interval is 1 minute
                if not args.direct:
                    time.sleep(random.randint(0, 60))
                # the eventcollector loop sets exit_code when issues ocure, no data processed doesn't mean
                # anything is wrong (it's just not of interest to Proofpoint).
                exit_code = 0
                s = requests.Session()
                for push_data in event_collector:
                    params = {
                        'timeout': 5,
                        'headers': {'Authorization': 'Bearer %s' % cnf.token},
                        'data': push_data.strip()
                    }
                    if args.insecure:
                        params['verify'] = False

                    r = s.post(args.endpoint, **params)
                    if r.status_code != 201:
                        syslog.syslog(
                            syslog.LOG_ERR,
                            'unexpected result from %s (http_code %s)' % (args.endpoint, r.status_code)
                        )
                        exit_code = -1
                        break
                    else:
                        try:
                            ujson.loads(r.text)
                        except ValueError:
                            syslog.syslog(syslog.LOG_ERR, 'telemetry unexpected response %s' % r.text[:256])
                            exit_code = -1
                            break
                if exit_code == 0:
                    # update timestamp, last record processed
                    telemetry_state.set_last_update(max_timestamp)
            else:
                # no data
                exit_code = 0
        else:
            syslog.syslog(syslog.LOG_ERR, 'directory %s missing' % args.log)
    else:
        syslog.syslog(syslog.LOG_ERR, 'telemetry token missing in %s' % args.config)


sys.exit(exit_code)