Setting up OPNsense mirror

[pongafence]

Hey all,

Have a lot of spare capacity in APAC region.  Wanting to setup OPNsense mirror.  What's best mode of contact?

[stejoo]

Bumping because I would also like to know how to set up a local mirror.

I'm going look up how to mirror FreeBSD and try syncing from the mirror URL I can find in /usr/local/etc/pkg/repos/OPNsense.conf but with rsync. If I manage to get it working I will report back.
I have also sent a message to franco asking for some help on this subject.

[edit]
I'm working from behind a proxy in an environment that aims to be self reliant and has limitations on what traffic can go out into the world. What I did was mirror a OPNsense mirror using rsync and configure my OPNsense to use it.

For setting up a a basic mirror for just the latest 19.1 this worked for me. This is on our local repository server (CentOS based).
First create the destination where the mirrored data should end up:

Code: [Select]

# mkdir /var/www/html/opnsense/FreeBSD:11:amd64After which you can give rsync a whirl. I my case I prepended the command with the RSYNC_PROXY environment flag because I need rsync to go through a proxy server instead of trying to contact the remote mirror directly. I also excluded the LibreSSL variant of OPNsense to save some space and only mirrored the 19.1. So I will have to change the mirror URL and paths when 19.7 is release. You can mirror a directory higher up to get the entire FreeBSD 11 based variants, or even higher to mirror all of OPNsense.
tl;dr: mirror with rsync and adapt the rsync command below to your needs.

Code: [Select]

# RSYNC_PROXY=proxy.svc.cal-net.nl:3128 \
rsync \
--archive \
--exclude=LibreSSL \
--exclude=libressl \
--hard-links \
--numeric-ids \
--stats \
"rsync://mirror.ams1.nl.leaseweb.net/opnsense/FreeBSD:11:amd64/19.1" \
"/var/www/html/opnsense/FreeBSD:11:amd64/"
After that's done log into OPNsense. Go to System > Firmware > Settings to change the Firmware Mirror setting.
Set it to "(other)" so you can specify your own custom URL and point that to your local mirror machine. Which needs to serve the contents over HTTP(S) (by means of Apache or nginx for example) so the rest of the network can access it easily.
I set my custom URL to: http://repositories.example.org/opnsense
Pressed the "Check for updates" and voila! updates appeared and I could install plugins

[jasonhollis]

Folks how hard/what would the process be to leverage https://hub.docker.com/r/seterrychen/apt-mirror-http-server as a means of just leveraging a docker container to take care of the mirror?

[kronenpj]

Thanks for starting this thread! My contribution is a slightly more flexible script which only mirrors the latest versions, e.g. FreeBSD:13:amd64/23.7, but all the patch levels underneath. It also doesn't clean up older copies.  Adding --delete-after --delete-excluded to the rsync command would take care of that auto-magically. Of course, change the values of MIRROR and LOCAL_MIRROR for your situation. I'm running this once per week in cron.

Code: [Select]

#!/usr/bin/env bash

MIRROR="mirror.wdc1.us.leaseweb.net"
LOCAL_MIRROR="/export/home2/mirror"
REPOPATH="opnsense"

# Determine latest major FreeBSD release offered
RELEASE=$(rsync rsync://${MIRROR}/${REPOPATH}/ |& \
grep FreeBSD | grep amd64 | tail -1 | awk '{print $5}')
REPOPATH="${REPOPATH}/$RELEASE"

# Determine latest opnsense release offered
REV=$(rsync rsync://${MIRROR}/${REPOPATH}/ |& \
grep -v snapshots | tail -1 | awk '{print $5}')

# Retrieve the repository, using as much local information as is available (-y)
rsync --archive -Py \
--exclude=LibreSSL \
--exclude=libressl \
--hard-links \
--numeric-ids \
--stats \
rsync://${MIRROR}/${REPOPATH}/$REV ${LOCAL_MIRROR}/${REPOPATH}