Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Topics - siga75

Pages: [1] 2

23.1 Legacy Series / gcloud command does not work anymore

« on: June 04, 2023, 10:38:05 am »

Hi all,

gcloud command does not work anymore, I noticed now since my certificates are expired, so it could be an issue of like 2 or 3 months ago. I use acme with DNS validation.

Nothing relevant in the logs. I also tried to truss it, but still not useful info

Code: [Select]

[root@myfw ~]# /usr/local/bin/gcloud dns record-sets list --name="www.signorini.ch." --type="A" -z "external-ch"
ERROR: gcloud crashed (AttributeError): 'NoneType' object has no attribute 'clean_version'

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics


[root@myfw ~]# gcloud info --run-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...failed.
ERROR: gcloud crashed (AttributeError): 'NoneType' object has no attribute 'clean_version'

If you would like to report this issue, please run the following command:
  gcloud feedback

To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics


[root@myfw ~]# pkg which /usr/local/bin/gcloud
/usr/local/bin/gcloud was installed by package google-cloud-sdk-431.0.0


[root@myfw ~]# pkg info google-cloud-sdk-431.0.0
google-cloud-sdk-431.0.0
Name           : google-cloud-sdk
Version        : 431.0.0
Installed on   : Fri May 26 11:26:15 2023 CEST
Origin         : net/google-cloud-sdk
Architecture   : FreeBSD:13:*
Prefix         : /usr/local
Categories     : net
Licenses       : APACHE20
Maintainer     : bofh@FreeBSD.org
WWW            : https://developers.google.com/cloud/sdk/
Comment        : Google Cloud SDK for Google Cloud Platform
Options        :
        BASH           : on
        ZSH            : on
Annotations    :
        repo_type      : binary
        repository     : OPNsense
Flat size      : 326MiB
Description    :
Google Cloud SDK contains tools and libraries that enable you to easily create
and manage resources on Google Cloud Platform, including App Engine, Compute
Engine, Cloud Storage, BigQuery, Cloud SQL, and Cloud DNS.

WWW: https://developers.google.com/cloud/sdk/


[root@myfw ~]# opnsense-version
OPNsense 23.1.9


[root@myfw ~]# find /usr/ /opt/ -type f | xargs grep -l clean_version 2>/dev/null
/usr/local/lib/perl5/5.32/CPAN/Meta/Converter.pm
/usr/local/google-cloud-sdk/lib/googlecloudsdk/core/util/platforms.py
/usr/local/google-cloud-sdk/lib/googlecloudsdk/core/util/__pycache__/platforms.cpython-39.pyc
/usr/local/google-cloud-sdk/lib/googlecloudsdk/core/__pycache__/transport.cpython-39.pyc
/usr/local/google-cloud-sdk/lib/googlecloudsdk/core/transport.py
^C

[root@myfw ~]# grep -10 clean_version /usr/local/google-cloud-sdk/lib/googlecloudsdk/core/util/platforms.py
      return not self.__lt__(other)

    @property
    def version(self):
      """Returns the operating system version."""
      if self == OperatingSystem.WINDOWS:
        return platform.version()
      return platform.release()

    @property
    def clean_version(self):
      """Returns a cleaned version of the operating system version."""
      version = self.version
      if self == OperatingSystem.WINDOWS:
        capitalized = version.upper()
        if capitalized in ('XP', 'VISTA'):
          return version
        if capitalized.startswith('SERVER'):
          # Allow Server + 4 digits for year.
          return version[:11].replace(' ', '_')


[root@myfw ~]# find /usr/local/google-cloud-sdk/  -type f | xargs grep -l NoneType  2>/dev/null
/usr/local/google-cloud-sdk/.install/.backup/lib/third_party/oauth2client/crypt.py
/usr/local/google-cloud-sdk/.install/.backup/lib/third_party/oauth2client/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/.install/.backup/lib/third_party/websocket/_core.py
/usr/local/google-cloud-sdk/.install/.backup/lib/third_party/apitools/base/py/encoding_helper.py
/usr/local/google-cloud-sdk/.install/.backup/lib/third_party/apitools/base/py/__pycache__/encoding_helper.cpython-37.pyc
/usr/local/google-cloud-sdk/.install/.backup/platform/bq/third_party/oauth2client_4_0/crypt.py
/usr/local/google-cloud-sdk/.install/.backup/platform/bq/third_party/oauth2client_4_0/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/.install/.backup/platform/gsutil/gslib/vendored/oauth2client/oauth2client/crypt.py
/usr/local/google-cloud-sdk/.install/.backup/platform/gsutil/gslib/vendored/oauth2client/oauth2client/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/.install/.backup/platform/gsutil/third_party/apitools/apitools/base/py/encoding_helper.py
/usr/local/google-cloud-sdk/.install/.backup/platform/gsutil/third_party/apitools/apitools/base/py/__pycache__/encoding_helper.cpython-37.pyc
/usr/local/google-cloud-sdk/lib/googlecloudsdk/api_lib/compute/iap_tunnel_lightweight_websocket.py
/usr/local/google-cloud-sdk/lib/third_party/oauth2client/crypt.py
/usr/local/google-cloud-sdk/lib/third_party/oauth2client/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/lib/third_party/oauth2client/__pycache__/crypt.cpython-38.pyc
/usr/local/google-cloud-sdk/lib/third_party/oauth2client/__pycache__/crypt.cpython-39.pyc
/usr/local/google-cloud-sdk/lib/third_party/apitools/base/py/encoding_helper.py
/usr/local/google-cloud-sdk/lib/third_party/apitools/base/py/__pycache__/encoding_helper.cpython-37.pyc
/usr/local/google-cloud-sdk/lib/third_party/apitools/base/py/__pycache__/encoding_helper.cpython-39.pyc
/usr/local/google-cloud-sdk/lib/third_party/apitools/base/py/__pycache__/encoding_helper.cpython-38.pyc
/usr/local/google-cloud-sdk/lib/third_party/websocket/_core.py
/usr/local/google-cloud-sdk/lib/third_party/jmespath/functions.py
/usr/local/google-cloud-sdk/platform/bq/third_party/oauth2client_4_0/crypt.py
/usr/local/google-cloud-sdk/platform/bq/third_party/oauth2client_4_0/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/platform/gsutil/gslib/vendored/oauth2client/oauth2client/crypt.py
/usr/local/google-cloud-sdk/platform/gsutil/gslib/vendored/oauth2client/oauth2client/__pycache__/crypt.cpython-37.pyc
/usr/local/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/encoding_helper.py
/usr/local/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/__pycache__/encoding_helper.cpython-37.pyc
/usr/local/google-cloud-sdk/platform/gsutil_py2/gslib/vendored/oauth2client/oauth2client/crypt.py
/usr/local/google-cloud-sdk/platform/gsutil_py2/third_party/apitools/apitools/base/py/encoding_helper.py

21.1 Legacy Series / suricata: cannot edit action anymore

« on: January 31, 2021, 03:13:17 pm »

wanted to change some pt.research rule from drop to alert, it's not possible anymore, both from the alert tab and the rules tab itself, the change is not taken

20.7 Legacy Series / firewall log does not work on 20.7.1

« on: August 13, 2020, 05:53:22 pm »

it's stuck

[root@myfw /var/log]# clog filter.log | tail -10
Aug 13 17:14:07 myfw filterlog[40478]: 20,,,0,igb1,match,block,in,4,0x0,,54,23849,0,none,6,tcp,83,108.177.119.188,172.17.33.50,5228,41725,31,PA,1266961685:1266961716,3492419179,261,,nop;nop;TS
Aug 13 17:14:07 myfw filterlog[40478]: 137,,,0,igb1,match,pass,in,4,0x2,0,127,28214,0,none,6,tcp,60,172.17.33.209,192.168.33.100,54702,443,0,SEC,4231299439,,64240,,mss;nop;wscale;sackOK;TS
Aug 13 17:14:07 myfw filterlog[40478]: 20,,,0,igb5,match,block,in,4,0x0,,64,1366,0,none,6,tcp,60,172.17.35.40,172.217.168.3,49807,443,0,S,2879250116,,65535,,mss;sackOK;TS;nop;wscale
Aug 13 17:14:07 myfw filterlog[40478]: 20,,,0,igb5,match,block,in,4,0x0,,64,12719,0,none,6,tcp,79,172.17.35.52,34.90.173.53,54792,443,39,PA,2241513410:2241513449,3581044844,4015,,
Aug 13 17:14:07 myfw filterlog[40478]: 20,,,0,igb5,match,block,in,4,0x0,,64,12720,0,none,6,tcp,79,172.17.35.52,34.90.173.53,54792,443,39,PA,2241513410:2241513449,3581044844,4015,,
Aug 13 17:14:08 myfw filterlog[40478]: 291,,,0,igb5,match,pass,in,4,0x0,,64,8328,0,none,17,udp,69,172.17.35.52,172.17.35.33,42936,53,49
Aug 13 17:14:08 myfw filterlog[40478]: 291,,,0,igb5,match,pass,in,4,0x0,,64,8329,0,none,17,udp,62,172.17.35.52,172.17.35.33,42800,53,42
Aug 13 17:14:08 myfw filterlog[40478]: 20,,,0,igb5,match,block,in,4,0x0,,64,12721,0,none,6,tcp,79,172.17.35.52,34.90.173.53,54792,443,39,PA,2241513410:2241513449,3581044844,4015,,
Aug 13 17:14:08 myfw filterlog[40478]: 291,,,0,igb5,match,pass,in,4,0x0,,64,8343,0,none,17,udp,68,172.17.35.52,172.17.35.33,47592,53,48
Aug 13 17:14:08 myfw filterlog[40478]: 291,,,0,igb5,match,pass,in,4,0x0,,64,8388,0,none,17,udp,65,172.17.35.52,172.17.35.33,33858,53,45
[root@myfw /var/log]# df -h .
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/rootfs 453G 14G 403G 3% /
[root@myfw /var/log]# date
Thu Aug 13 17:51:55 CEST 2020
[root@myfw /var/log]# ls -l filter.log
-rw------- 1 root wheel 10000000 Aug 13 17:14 filter.log

20.1 Legacy Series / acme with gcloud DNS - python error

« on: July 05, 2020, 10:56:10 am »

gcloud validation does not work anymore since the last 20.1.8 update

Also manually using gcloud command does not works:

[root@myfw ~]# gcloud dns record-sets list -z internal
ERROR: gcloud failed to load: No module named _sqlite3
gcloud_main = _import_gcloud_main()
import googlecloudsdk.gcloud_main
from googlecloudsdk.api_lib.iamcredentials import util as iamcred_util
from googlecloudsdk.core.credentials import http as http_creds
from googlecloudsdk.core.credentials import creds as core_creds
import sqlite3
from dbapi2 import *
from _sqlite3 import *

This usually indicates corruption in your gcloud installation or problems with your Python interpreter.

Please verify that the following is the path to a working Python 2.7 or 3.5+ executable:
/usr/local/bin/python2

If it is not, please set the CLOUDSDK_PYTHON environment variable to point to a working Python 2.7 or 3.5+ executable.

If you are still experiencing problems, please reinstall the Cloud SDK using the instructions here:
https://cloud.google.com/sdk/

Adding the env var in /etc/login.conf
default:\
29 :setenv=MAIL=/var/mail/$,BLOCKSIZE=K,CLOUDSDK_PYTHON=/usr/local/bin/python3:\

And rebuilding makes the command line to correctly works

root@myfw:~ # cap_mkdb /etc/login.conf

Still acme validation does not works, I tried to add

export CLOUDSDK_PYTHON=/usr/local/bin/python3

in the following scripts, none of them solved the issue

/usr/local/bin/gcloud
/usr/local/sbin/acme.sh
/usr/local/share/examples/acme.sh/dnsapi/dns_gcloud.sh

20.1 Legacy Series / let's encrypt SFTP error

« on: June 17, 2020, 07:54:49 pm »

I get this error when clicking "show identity", and if I use that key the connection does not work

Warning: "continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? in /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php on line 302 restrict,command="internal-sftp",from="192.168.xxx.xxx" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDrAvii8lFJurCX/3boJ/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/w73BLYR4CbqiFJQpGSrHa9jfjMKow5XTPst1ZeoipeixmJZOnjUc4q/+bnXwLLZIrlz5+0Q5xc/jmBRATcjBbaHRwHJShf9fQPko/gcLqHEvbqED2yn0NpqpsS2vxQuvsVNlEwIXAB/fe1zYMht2iOV/nmSw== root@xxxxxxxxxxxxx

Intrusion Detection and Prevention / out of memory

« on: May 31, 2020, 04:51:33 pm »

I just installed Veem for baking up my gaming laptop, which is in WAN in respect of OPNsense, to a NAS in a network with jumbo frame enabled.

The backup is like 300GB and suricata, which is configured to IPS the WAN, keep eating more and more memory.

My system has 16GB RAM and 8GB swap, it freeze for having finished the memory. I can't even login since he can't fork new processes, I had to brutally power it off

In normal operation I have 12GB of free mem.

I don't know if it's a new issue since I never did such big transfers in WAN, or if jumbo frames play something here.

Tutorials and FAQs / update my public IP address in the A Record of Google Cloud DNS

« on: May 23, 2020, 10:10:25 am »

It could be useful for someone, so I just put it here

[root@myfw ~]# cat gcloud-dns-updatemyip
#!/bin/sh

######################################################################
#
# update my public IP address in the A Record of Google Cloud DNS
#
# 20200523 - siga: creation
#

######################################################################
# set constants

SCRIPTNAME="$( basename $0 )"
LOGDIR="/var/log"
LOGFILE="${LOGDIR}/${SCRIPTNAME}.log"
ARECORD="www.signorini.ch."

######################################################################
# set functions

log()
{
message="$1"
date="$( date +%Y%m%d-%H%M%S )"
echo "${date}: $message" | tee -a "$LOGFILE"
}

######################################################################
# main

oldip=$( /usr/local/bin/gcloud dns record-sets list --name="$ARECORD" --type="A" -z "external-ch" | awk -v NAME="$ARECORD" '$1==NAME {print $4}' )
newip=$( /usr/local/bin/curl -s ifconfig.me )

if [ "$oldip" != "$newip" ]; then
log "my ip is changed: oldip: $oldip - newip: $newip"
/usr/local/bin/gcloud dns record-sets transaction start --zone="external-ch" | tee -a "$LOGFILE"
/usr/local/bin/gcloud dns record-sets transaction remove "$oldip" --zone="external-ch" --name="$ARECORD" --type="A" --ttl="300" | tee -a "$LOGFILE"
/usr/local/bin/gcloud dns record-sets transaction add "$newip" --zone="external-ch" --name="$ARECORD" --type="A" --ttl="300" | tee -a "$LOGFILE"
/usr/local/bin/gcloud dns record-sets transaction execute --zone=external-ch | tee -a "$LOGFILE"
else
log "myip is not changed: $oldip"
fi

exit 0

General Discussion / what come first: pf or sensei?

« on: February 15, 2020, 09:25:54 am »

my logic tell me it should be pf first then sensei, but based on some observation it looks like the opposite.

just to be sure since I see a strange behavior in sensei filtering and I would like to know it before deeper investigation

General Discussion / NAT issue using aliases

« on: January 18, 2020, 06:27:44 pm »

I have a port forwarding defined for let go out only some ip addresses, through squid.

The alias used (https_www_proxied_hosts) does not seems to contains everything defined

See attachments for details

I defined a cron job to run every 5 minutes to reload the aliases

See here the content of the pf table for alias google_networks

[root@myfw ~]# pfctl -t google_networks -T show
35.190.247.0/24
35.191.0.0/16
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
108.177.8.0/21
108.177.96.0/19
130.211.0.0/22
172.217.0.0/19
172.217.32.0/20
172.217.128.0/19
172.217.160.0/20
172.217.192.0/19
172.253.56.0/21
172.253.112.0/20
173.194.0.0/16
209.85.128.0/17
216.58.192.0/19
216.239.32.0/19

That should be included in this table, but completely missing:

[root@myfw ~]# pfctl -t https_www_proxied_hosts -T show
23.23.73.124
50.19.218.16
54.204.26.223
54.225.71.235
54.225.169.250
54.235.203.7
54.235.220.229
54.243.147.226
104.17.172.102
104.18.48.62
104.18.49.62
104.18.206.87
104.24.110.187
104.24.111.187
104.31.90.50
104.31.91.50
104.131.209.4
108.171.202.195
108.171.202.203
108.171.202.211
172.217.168.4
172.217.168.10
172.217.168.14
172.217.168.42
172.217.168.45
172.217.168.46
172.217.168.78
198.143.164.0/24
198.143.164.251
198.143.164.252
216.58.215.225
216.58.215.234
2606:4700::6811:ac66
2606:4700::6812:ce57
2606:4700:3033::6818:6fbb
2606:4700:3035::6812:313e
2606:4700:3037::6812:303e
2606:4700:3037::6818:6ebb
2606:4700:3037::681f:5a32
2606:4700:3037::681f:5b32
2607:f2d8:4010:8::2
2607:f2d8:4010:b::2
2607:f2d8:4010:c::2
2a00:1450:400a:800::2001
2a00:1450:400a:800::200a
2a00:1450:400a:801::2004
2a00:1450:400a:801::200e
2a00:1450:400a:802::200a
2a00:1450:400a:802::200d
2a00:1450:400a:802::200e

Any idea what's the issue here? I saw in the forum in the past there was a bug when using aliases for NAT rules, and here I use an external alias (google_networks) that may complicate the stuff. Could this be the issue?

THX

General Discussion / gpsd

« on: January 18, 2020, 11:10:36 am »

does OPNsense use gpsd for GPS timing? I see the package is not installed and not in the repo

root@myfw:/usr/ports/astro/gpsd # pkg install gpsd
Updating OPNsense repository catalogue...
OPNsense repository is up to date.
Updating SunnyValley repository catalogue...
SunnyValley repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'gpsd' have been found in the repositories

Anyway I found this, which does not come from a package:
root@myfw:/usr/ports/astro/gpsd # cat distinfo
TIMESTAMP = 1516146278
SHA256 (gpsd-3.17.tar.gz) = 68e0dbecfb5831997f8b3d6ba48aed812eb465d8c0089420ab68f9ce4d85e77a
SIZE (gpsd-3.17.tar.gz) = 8755304
root@myfw:/usr/ports/astro/gpsd # pkg info|grep gps
root@myfw:/usr/ports/astro/gpsd #

root@myfw:/usr/ports/astro/gpsd # pkg which /usr/ports/astro/gpsd/distinfo
/usr/ports/astro/gpsd/distinfo was not found in the database

Also with a ps aux I can't find a gps daemon, so how does it works?

I just ordered a u-blox zed-f9t and would like to know if gpsd 3.19 (or 3.20) will be part of the 20.1 release

General Discussion / chronyd

« on: January 10, 2020, 08:05:56 am »

is it planned to use chronyd instead of ntpd on 20.1? chrony is much better in general, but in particular for machine with a bad hardware clock, or for virtual machines

unrelated to that it would be nice to have, on ntp conf:
- configure peers and not only servers
- minpoll (some ntp servers are not happy with too frequent requests)
- GPS/PPS conf currently is or prefer or noselect, it should be possible to have nothing specified

cheers

Intrusion Detection and Prevention / ET telemetry issue?

« on: January 05, 2020, 08:04:36 pm »

Am I the only one?

[root@myfw /var/log]# clog system.log | grep telemetry | tail
Jan 5 19:55:01 myfw /send_telemetry.py: telemetry data collected 35 records in 0.24 seconds @2020-01-05 18:39:22.841777
Jan 5 19:55:31 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)
Jan 5 19:56:01 myfw /send_telemetry.py: telemetry data collected 35 records in 0.09 seconds @2020-01-05 18:39:22.841777
Jan 5 19:56:36 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)
Jan 5 19:57:01 myfw /send_telemetry.py: telemetry data collected 35 records in 0.10 seconds @2020-01-05 18:39:22.841777
Jan 5 19:57:57 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)
Jan 5 19:58:00 myfw /send_telemetry.py: telemetry data collected 35 records in 0.11 seconds @2020-01-05 18:39:22.841777
Jan 5 19:58:23 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)
Jan 5 19:59:01 myfw /send_telemetry.py: telemetry data collected 35 records in 0.36 seconds @2020-01-05 18:39:22.841777
Jan 5 19:59:04 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)

[root@myfw ~]# /usr/local/opnsense/scripts/etpro_telemetry/send_telemetry.py
[root@myfw ~]# echo $?
255

[root@myfw /var/log]# clog system.log | grep "http_code 500" | head -1
Jan 4 03:23:42 myfw /send_heartbeat.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/telemetry (http_code 500)
[root@myfw /var/log]# clog system.log | grep "http_code 500" | tail -1
Jan 5 19:57:57 myfw /send_telemetry.py: unexpected result from https://opnsense.emergingthreats.net/api/v1/event (http_code 500)

Tutorials and FAQs / create an alias for google networks

« on: December 28, 2019, 09:26:13 am »

Any comment is welcome since I am not sure it's the best way to achieve it or if it has some drawbacks

tl;dr

[root@myfw ~]# cat google-nets
#!/bin/sh
dig @8.8.8.8 +noall +answer +short +dnssec +tcp _spf.google.com txt | tr ' ' '\n' | awk -F: '$1=="include" {print $2}' | while read blocks
do
dig @8.8.8.8 +noall +answer +short +dnssec +tcp "$blocks" txt | tr ' ' '\n' | awk -F: '$1=="ip4" {print $2}'
done | xargs pfctl -t google_networks -T replace

[root@myfw ~]# cat /usr/local/etc/cron.d/custom-pf-tables.cron
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
#minute hour mday month wday who command
40 4 * * * root /root/google-nets

some more details:
https://www.signorini.ch/content/opnsense-create-pftable-for-google-networks

General Discussion / captive portal - cannot disable "enforce local group"

« on: December 24, 2019, 10:05:34 am »

I cannot select "none", if I do that, when I save it come back to "admin", I only can select others groups, I tried creating a wlan_guests group, but then how I am supposed to link vouchers to that group?

19.7 Legacy Series / 19.7.8 looks good

« on: December 18, 2019, 08:12:07 pm »

just updated, so far so good

THANKS!

Pages: [1] 2