Messages

Sure.

The aim is to deploy OPNsense on hetzner and route all traffic (in particular public) through it.

Each node will just have a private interface (no public ip).

In other words:

- node > OPNsense > internet

At the moment I am configuring the node via cloud-init:

Code Select Expand

#cloud-config
users:
  - name: admin
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - <key>
write_files:
  - path: /etc/resolv.conf
    content: |
      nameserver 10.0.0.10
  - path: /etc/network/interfaces
    content: |
      auto enp7s0
      iface enp7s0 inet static
        address 10.0.0.20
        netmask 255.255.255.0
        network 10.0.0.0
        gateway 10.0.0.10
        dns-nameservers 10.0.0.10
  - path: /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
    content: |
      network:
        config: disabled
runcmd:
  - ip route add default via 10.0.0.1


Ok, are you suggesting to set default route for the node to 10.0.0.10 (OPNSense private ip)?

If I try, get:

Code Select Expand

$ sudo ip route add default via 10.0.0.10
Error: Nexthop has invalid gateway.


I am sorry, I am trying to understand.

But if the default route would be a problem, I should not see the ping request arriving to OPNsense via tcpdump or the connection via the WebUI.

Am I correct?

Quoting your first post (this is on your server!):

Code Select Expand

$ ip route
default via 10.0.0.1 dev enp7s0
10.0.0.0/24 via 10.0.0.1 dev enp7s0
10.0.0.1 dev enp7s0 scope link
169.254.169.254 via 10.0.0.1 dev enp7s0

How do you expect OpnSense to do anything if packets do not even touch it?

From my understanding, the node has be to set to the hetzner gateway (10.0.0.1), otherwise the private traffic cannot be routed inside the virtual private network.

From my ping/dns/tcpdump experiment, packets are actually arriving at OPNsense but for some reason OPNsense is not "responding" the outbound request from the node.

Thanks for the reply.

To clarify:

* LAN network CIDR is 10.0.0.0/24.
* OPNsense IP is 10.0.0.10
* 10.0.0.1 is Hetzner gateway required to route traffic in the private network

I have set out a hetzner network route: destination 0.0.0.0/0, gateway 10.0.0.10.

Similar to https://community.hetzner.com/tutorials/how-to-route-cloudserver-over-private-network-using-pfsense-and-hcnetworks

Regarding outbound NAT, I tried:

* default automatic
* hybrid with an additional LAN, any source, any destination, translate to WAN interface

no difference 

Hi,

I am trying to familiarize with OPNsense by deploying it on Hetzner.

I have:

- router: a server (10.0.0.10) with OPNsense installed on it, I can access the WebUI/ssh from the WAN interface.
- node: another server (10.0.0.20, no public IP) with Debian 12:

Code Select Expand

$ cat /etc/resolv.conf
nameserver 10.0.0.10

$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP group default qlen 1000
    link/ether 86:00:00:dd:15:64 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.20/32 brd 10.0.0.20 scope global dynamic enp7s0
       valid_lft 86200sec preferred_lft 86200sec
    inet6 fe80::8400:ff:fedd:1564/64 scope link
       valid_lft forever preferred_lft forever

$ ip route
default via 10.0.0.1 dev enp7s0
10.0.0.0/24 via 10.0.0.1 dev enp7s0
10.0.0.1 dev enp7s0 scope link
169.254.169.254 via 10.0.0.1 dev enp7s0

I can ssh to the node via the router:

Code Select Expand

ssh -J root@$IP_PUBLIC 10.0.0.20

I can ping the router from the node:

Code Select Expand

$ ssh -J root@$IP_PUBLIC admin@10.0.0.20 ping 10.0.0.10
PING 10.0.0.10 (10.0.0.10) 56(84) bytes of data.
64 bytes from 10.0.0.10: icmp_seq=1 ttl=63 time=0.955 ms

I can resolve dns:

Code Select Expand

$ ssh -J root@$IP_PUBLIC admin@10.0.0.20 dig example.com

; <<>> DiG 9.18.28-1~deb12u2-Debian <<>> example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50112
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;example.com. IN A

;; ANSWER SECTION:
example.com. 300 IN A 96.7.128.198
example.com. 300 IN A 23.192.228.80
example.com. 300 IN A 23.192.228.84
example.com. 300 IN A 23.215.0.136
example.com. 300 IN A 23.215.0.138
example.com. 300 IN A 96.7.128.175

;; Query time: 268 msec
;; SERVER: 10.0.0.10#53(10.0.0.10) (UDP)
;; WHEN: Wed Jan 22 08:23:35 UTC 2025
;; MSG SIZE  rcvd: 136


But the node does not have access to the internet.

For example if I try to ping 1.1.1.1 from the node, via tcpdump on the router, I can see the request but no response:

Code Select Expand

root@OPNsense:~ # tcpdump -i vtnet1 not port 22
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vtnet1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
08:25:56.061477 IP 10.0.0.20 > one.one.one.one: ICMP echo request, id 904, seq 1, length 64
08:25:57.064052 IP 10.0.0.20 > one.one.one.one: ICMP echo request, id 904, seq 2, length 64

From the WebUI > Reporting > Traffic > Top talkers, I can see the one.one.one.one.(1.1.1.1) request.

Could you suggest where I can go to get more insight on why a request is blocked/...?

Thanks

Thanks for your reply.

In the end the problem was that: even if the serial console was showing the public IP with DHCP, switching to static made it work:

Code Select Expand

  <interfaces>
    <wan>
      <enable>1</enable>
      <if>vtnet0</if>
      <ipaddr>IP_PUBLIC</ipaddr>
      <subnet>32</subnet>
      <gateway>WAN_GW</gateway>
      <blockpriv>1</blockpriv>
      <blockbogons>1</blockbogons>
    </wan>
...
  <OPNsense>
    <Gateways version="1.0.0">
      <gateway_item uuid="179b59be-87d3-4f2a-b866-2664c7f31577">
        <disabled>0</disabled>
        <name>WAN_GW</name>
        <descr>HCloud default gateway</descr>
        <interface>wan</interface>
        <ipprotocol>inet</ipprotocol>
        <gateway>172.31.1.1</gateway>
        <defaultgw>1</defaultgw>
        <fargw>1</fargw>
        <monitor_disable>1</monitor_disable>
        <monitor_noroute/>
        <monitor/>
        <force_down/>
        <priority>255</priority>
        <weight>1</weight>
        <latencylow/>
        <latencyhigh/>
        <losslow/>
        <losshigh/>
        <interval/>
        <time_period/>
        <loss_interval/>
        <data_length/>
      </gateway_item>
    </Gateways>
  </OPNsense>


Thanks for your reply.

To WAN is correctly assigned the public IP of the instance.

Apart the root/password login, I am not even able to ping or connect to ssh.

I do not rely care about password/security at this stage (I just want to make it run).

Anyway the instance has a firewall rule that allows traffic in only via my public IP.

After setting:

Code Select Expand

      <blockpriv>0</blockpriv>
      <blockbogons>0</blockbogons>

Still no ssh:

Code Select Expand

$ ssh -v root@<public_ip>
OpenSSH_9.7p1, OpenSSL 3.3.2 3 Sep 2024
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 22: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: Connecting to <public_ip> [<public_ip>] port 22.


Hi,

I would like to try OPNsense on hcloud for test purpose.

Starting from a FreeBSD instance, I am installing OPNsense via the script:

Code Select Expand

sh ./opnsense-bootstrap.sh.in -r 24.7 -y

and copy the following file to /conf/config.xml:

Code Select Expand

<?xml version="1.0"?>
<opnsense>
  <trigger_initial_wizard/>

  <theme>opnsense</theme>

  <system>
    <optimization>normal</optimization>
    <hostname>OPNsense</hostname>
    <domain>localdomain</domain>
    <dnsallowoverride>1</dnsallowoverride>
    <group>
      <name>admins</name>
      <description><![CDATA[System Administrators]]></description>
      <scope>system</scope>
      <gid>1999</gid>
      <member>0</member>
      <priv>page-all</priv>
    </group>
    <user>
      <name>root</name>
      <descr><![CDATA[System Administrator]]></descr>
      <scope>system</scope>
      <groupname>admins</groupname>
      <password>$6$DEHx8zOr5VVhjFwz$pS/NcRnn03fpFxbR2xP0gkfUSdSDA0FZxRF3zCn09qTFF604k800paZCNnxrMwElH8gAsPK8GDHBFgOX32YI2/</password>
      <uid>0</uid>
    </user>
    <nextuid>2000</nextuid>
    <nextgid>2000</nextgid>
    <webgui>
      <protocol>https</protocol>
      <nohttpreferercheck>1</nohttpreferercheck>
    </webgui>
    <disablenatreflection>yes</disablenatreflection>
    <usevirtualterminal>1</usevirtualterminal>
    <disableconsolemenu/>
    <disablevlanhwfilter>1</disablevlanhwfilter>
    <disablechecksumoffloading>1</disablechecksumoffloading>
    <disablesegmentationoffloading>1</disablesegmentationoffloading>
    <disablelargereceiveoffloading>1</disablelargereceiveoffloading>
    <pf_share_forward>1</pf_share_forward>
    <lb_use_sticky>1</lb_use_sticky>
    <ssh>
      <group>admins</group>
      <enabled>enabled</enabled>
    </ssh>
  </system>

  <interfaces>
    <wan>
      <enable>1</enable>
      <if>vtnet0</if>
      <mtu/>
      <ipaddr>dhcp</ipaddr>
      <subnet/>
      <gateway/>
      <blockpriv>1</blockpriv>
      <blockbogons>1</blockbogons>
      <dhcphostname/>
      <media/>
      <mediaopt/>
    </wan>
    <lan>
      <enable>1</enable>
      <if>vtnet1</if>
      <ipaddr>dhcp</ipaddr>
      <subnet/>
      <media/>
      <mediaopt/>
    </lan>
  </interfaces>

  <dhcpd>
    <lan>
      <enable/>
      <range>
        <from>192.168.1.100</from>
        <to>192.168.1.199</to>
      </range>
    </lan>
  </dhcpd>

  <unbound>
    <enable>1</enable>
  </unbound>

  <nat>
    <outbound>
      <mode>automatic</mode>
    </outbound>
  </nat>

  <filter>
    <rule>
      <type>pass</type>
      <interface>lan</interface>
      <ipprotocol>inet</ipprotocol>
      <statetype>keep state</statetype>
      <descr><![CDATA[Allow public ICMP ping to this firewall from LAN]]></descr>
      <protocol>icmp</protocol>
      <icmptype>echoreq</icmptype>
      <source>
        <any>1</any>
      </source>
      <destination>
        <network>(self)</network>
      </destination>
    </rule>

    <rule>
      <type>pass</type>
      <interface>wan</interface>
      <ipprotocol>inet</ipprotocol>
      <statetype>keep state</statetype>
      <descr><![CDATA[Allow public ICMP ping to this firewall from WAN]]></descr>
      <protocol>icmp</protocol>
      <icmptype>echoreq</icmptype>
      <source>
        <any>1</any>
      </source>
      <destination>
        <network>(self)</network>
      </destination>
    </rule>

    <rule>
      <type>pass</type>
      <interface>wan</interface>
      <ipprotocol>inet4</ipprotocol>
      <statetype>keep state</statetype>
      <descr><![CDATA[CAUTION: Allow public remote access to this firewall via SSH]]></descr>
      <protocol>tcp</protocol>
      <source>
        <any>1</any>
      </source>
      <destination>
        <network>(self)</network>
        <port>22</port>
      </destination>
    </rule>

    <rule>
      <type>pass</type>
      <interface>wan</interface>
      <ipprotocol>inet4</ipprotocol>
      <statetype>keep state</statetype>
      <descr><![CDATA[CAUTION: Allow public remote access to this firewall via HTTP]]></descr>
      <protocol>tcp</protocol>
      <source>
        <any>1</any>
      </source>
      <destination>
        <network>(self)</network>
        <port>80</port>
      </destination>
    </rule>

    <rule>
      <type>pass</type>
      <interface>wan</interface>
      <ipprotocol>inet4</ipprotocol>
      <statetype>keep state</statetype>
      <descr><![CDATA[CAUTION: Allow public remote access to this firewall via HTTPS]]></descr>
      <protocol>tcp</protocol>
      <source>
        <any>1</any>
      </source>
      <destination>
        <network>(self)</network>
        <port>443</port>
      </destination>
    </rule>
  </filter>

</opnsense>


Seems that the config file is read, as I am able to login, via the serial console, with root and the password above.

From the serial console I can see that:

* LAN (vtnet1) gets an ip via dhcp
* WAN (vtnet0) gets an ip via dhcp (the public one)

But I am not able to ping/ssh/access the web ui.

Am I missing something?

Thanks