[Solved] Maltrail plugin just stopped detecting anything

[Taomyn]

I've had Maltrail running pretty well from 5th August to 12th August, but since then it's made zero detections.


The service is running, there is nothing in its error log. I restarted the firewall and still the same. I'm using the two test examples from the Maltrail readme:

Code: [Select]

nslookup morphed.ru.
ping 136.161.101.53

Neither gets picked up.


I'm on OPNsense v19.7.2 and the plugin is v1.0 - Maltrail is monitoring the WAN interface.

[mimugmail]

Can you post maltrail.conf please?

[Taomyn]

Can you post maltrail.conf please?

Which one? When I search I see two, one which looks like a standard one and another with OPNsense tokens all over the place.

[mimugmail]

/usr/local/share/maltrail/maltrail.conf

[Taomyn]

/usr/local/share/maltrail/maltrail.conf

# [Server]
HTTP_ADDRESS 192.168.1.1
HTTP_PORT 8338
USE_SSL false




DISABLE_LOCAL_LOG_STORAGE false


SENSOR_NAME $HOSTNAME
CUSTOM_TRAILS_DIR /usr/local/maltrail/trails/custom/
PROCESS_COUNT $CPU_CORES
DISABLE_CPU_AFFINITY false
USE_FEED_UPDATES true
DISABLED_FEEDS turris, ciarmy, policeman, myip
UPDATE_PERIOD
USE_SERVER_UPDATE_TRAILS false
USE_HEURISTICS true
CHECK_MISSING_HOST false
CHECK_HOST_DOMAINS false
SHOW_DEBUG false
LOG_DIR /var/log/maltrail
MONITOR_INTERFACE pppoe0
CAPTURE_BUFFER 10%
CAPTURE_FILTER udp or icmp or (tcp and (tcp[tcpflags] == tcp-syn or port 80 or port 1080 or port 3128 or port 8000 or port 8080 or port 8118))
USERS
    admin:CHANGED:2000:0.0.0.0/0                        # changeme!

[mimugmail]

Can you try lan or physical wan, maybe a problem with pppoe

[Taomyn]

Can you try lan or physical wan, maybe a problem with pppoe

I will when I get to my destination and can remote into my network, but seeing as it's been fine for many days and nothing else changed I'm not sure if that will make a difference. Plus the PPPoE is my WAN it's how I connect to my ISP.


I'll keep you updated.

[mimugmail]

You can also try to start sensor manually so it runs in foreground

[Taomyn]

You can also try to start sensor manually so it runs in foreground

How do I do that and I presume I need to stop the service version first?

[mimugmail]

/usr/local/etc/rc.d/opnsense-maltrailsensor stop
python2.7 /usr/local/share/maltrail/sensor.py

[Taomyn]

/usr/local/etc/rc.d/opnsense-maltrailsensor stop
python2.7 /usr/local/share/maltrail/sensor.py

Done

Code: [Select]

root@bart:~ # /usr/local/etc/rc.d/opnsense-maltrailsensor stop
Stopping maltrailsensor.
Waiting for PIDS: 41882.
root@bart:~ # python2.7 /usr/local/share/maltrail/sensor.py
Maltrail (sensor) #v0.13.26


[i] using configuration file '/usr/local/share/maltrail/maltrail.conf'
[i] using '/var/log/maltrail' for log storage
[?] at least 384MB of free memory required
[i] using '/root/.maltrail/trails.csv' for trail storage
[i] updating trails (this might take a while)...
 [o] 'https://data.netlab.360.com/feeds/dga/chinad.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/conficker.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/cryptolocker.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/gameover.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/locky.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/necurs.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/tofsee.txt'
 [o] 'https://data.netlab.360.com/feeds/dga/virut.txt'
 [o] 'https://www.abuseipdb.com/statistics'
 [o] 'https://reputation.alienvault.com/reputation.generic'
 [o] 'https://cybercrime-tracker.net/ccam.php'
 [o] 'https://www.badips.com/get/list/any/2?age=7d'
 [o] 'https://osint.bambenekconsulting.com/feeds/c2-dommasterlist-high.txt'
 [o] 'https://osint.bambenekconsulting.com/feeds/dga-feed.txt'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/bitcoin_nodes_1d.ipset'
 [o] 'https://raw.githubusercontent.com/stamparm/blackbook/master/blackbook.csv'
 [o] 'https://lists.blocklist.de/lists/all.txt'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/botscout_1d.ipset'
 [o] 'http://danger.rulez.sk/projects/bruteforceblocker/blist.php'
 [o] 'https://raw.githubusercontent.com/fox-it/cobaltstrike-extraneous-space/master/cobaltstrike-servers.csv'
 [o] 'https://www.cruzit.com/xxwbl2txt.php'
 [o] 'https://cybercrime-tracker.net/all.php'
 [o] 'https://dataplane.org/*.txt'
 [o] 'https://isc.sans.edu/feeds/suspiciousdomains_Low.txt'
 [o] 'https://feeds.dshield.org/top10-2.txt'
 [o] 'https://rules.emergingthreats.net/open/suricata/rules/botcc.rules'
 [o] 'https://rules.emergingthreats.net/open/suricata/rules/compromised-ips.txt'
 [o] 'https://rules.emergingthreats.net/open/suricata/rules/emerging-dns.rules'
 [o] 'https://feodotracker.abuse.ch/blocklist/?download=domainblocklist'
 [o] 'https://feodotracker.abuse.ch/blocklist/?download=ipblocklist'
 [o] 'https://blocklist.greensnow.co/greensnow.txt'
 [o] 'https://raw.githubusercontent.com/Neo23x0/signature-base/master/iocs/otx-c2-iocs.txt'
 [o] 'https://raw.githubusercontent.com/gwillem/magento-malware-scanner/master/rules/burner-domains.txt'
 [o] 'https://malc0de.com/bl/ZONES'
 [o] 'https://www.malwaredomainlist.com/hostslist/hosts.txt'
 [o] 'http://malwaredomains.lehigh.edu/files/domains.txt'
 [o] 'https://www.maxmind.com/en/high-risk-ip-sample-list'
 [o] 'https://raw.githubusercontent.com/Hestat/minerchk/master/hostslist.txt'
 [o] 'https://www.nothink.org/blacklist/blacklist_malware_irc.txt'
 [o] 'https://openphish.com/feed.txt'
 [o] 'https://palevotracker.abuse.ch/blocklists.php?download=combinedblocklist'
 [o] 'https://cybercrime-tracker.net/ccpmgate.php'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/proxylists_1d.ipset'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/proxyrss_1d.ipset'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/proxyspy_1d.ipset'
 [o] 'https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt'
 [o] 'https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt'
 [o] 'https://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/ri_web_proxies_30d.ipset'
 [o] 'https://report.cs.rutgers.edu/DROP/attackers'
 [o] 'https://sblam.com/blacklist.txt'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/socks_proxy_7d.ipset'
 [o] 'https://sslbl.abuse.ch/blacklist/sslipblacklist.csv'
 [o] 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/sslproxies_1d.ipset'
 [o] 'https://www.talosintelligence.com/feeds/ip-filter.blf'
 [o] 'https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1'
 [o] 'https://torstatus.blutmagie.de/ip_list_all.php/Tor_ip_list_ALL.csv'
 [o] 'https://github.com/JR0driguezB/malware_configs'
 [o] 'https://urlhaus.abuse.ch/downloads/text/'
 [o] 'http://www.urlvir.com/export-hosts/'
 [o] 'http://www.voipbl.org/update/'
 [o] 'http://vxvault.net/URL_List.php'
 [o] 'https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist'
 [o] 'https://zeustracker.abuse.ch/blocklist.php?download=badips'
 [o] 'https://zeustracker.abuse.ch/monitor.php?filter=all'
 [o] 'https://zeustracker.abuse.ch/blocklist.php?download=compromised'
 [o] '(static)'
 [o] '(custom)'
[x] something went wrong during remote data retrieval ('(custom)')
[i] update finished
[i] trails stored to '/root/.maltrail/trails.csv'
[i] updating ipcat database...
[i] opening interface 'pppoe0'
[i] setting capture filter 'udp or icmp or (tcp and (tcp[tcpflags] == tcp-syn or port 80 or port 1080 or port 3128 or port 8000 or port 8080 or port 8118))'
[i] preparing capture buffer...
[i] creating 3 more processes (out of total 4)
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/local/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/local/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'


Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/local/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/local/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'


Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/local/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/local/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'


Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 1071, in run
    self.finished.wait(self.interval)
  File "/usr/local/lib/python2.7/threading.py", line 614, in wait
    self.__cond.wait(timeout)
  File "/usr/local/lib/python2.7/threading.py", line 349, in wait
    endtime = _time() + timeout
TypeError: unsupported operand type(s) for +: 'float' and 'str'


[?] please install 'schedtool' for better CPU scheduling
[o] running...


[Taomyn]

Issue opened on Github https://github.com/opnsense/plugins/issues/1470

[Taomyn]

Issue fixed in upcoming patch - thank-you

[firewall]

I'm experiencing the same behavior on 19.7.5_5, Maltrail version 1.2.  Works fine for a day or 2 then just craps out.  Any suggestions?

maltrail.conf is as follows:

Code: [Select]

# [Server]
HTTP_ADDRESS 10.10.10.1
HTTP_PORT 8338
USE_SSL false


DISABLE_LOCAL_LOG_STORAGE false

SENSOR_NAME $HOSTNAME
CUSTOM_TRAILS_DIR /usr/local/maltrail/trails/custom/
PROCESS_COUNT $CPU_CORES
DISABLE_CPU_AFFINITY false
USE_FEED_UPDATES true
DISABLED_FEEDS turris, ciarmy, policeman, myip
UPDATE_PERIOD 86400
USE_SERVER_UPDATE_TRAILS false
USE_HEURISTICS true
CHECK_MISSING_HOST false
CHECK_HOST_DOMAINS false
SHOW_DEBUG false
LOG_DIR /var/log/maltrail
MONITOR_INTERFACE igb2,ovpns2,ovpnc4,ovpnc3,ovpnc1,igb1,igb3
CAPTURE_BUFFER 10%
CAPTURE_FILTER udp or icmp or (tcp and (tcp[tcpflags] == tcp-syn or port 80 or port 1080 or port 3128 or port 8000 or port 8080 or port 8118))
USERS
    admin:5dc7ddttttt9f87c18ce4db9ttttte5a94c7c88tttttd655325ttttt5698c336:2000:0.0.0.0/0                        # changeme!



[mimugmail]

You mean with pppoe?