Good evening,
I am trying to handle and route a bunch of networks differently through my OPNSense firewall and I was glad to see that it supports, in theory, url tables (https://wiki.opnsense.org/manual/aliases.html#url-tables) but my problem is that they are not fetched and imho incorrectly are logged as returning a 500 error.
First things first, I tried one URL table(s) alias with one URL and with multiple URLs, it happens either way. By it I mean, URLs that work and whose server answer the requests coming from my opnsense router with the correct content and HTTP 200, OPNSense logs as errorneous like this:
2021-10-25T20:06:23 /update_tables.py[33526] error fetching alias url https://workers.mycloudflaredomain.com/?asn=24424&ipv4=yes&details=yes
2021-10-25T20:06:23 /update_tables.py[33526] error fetching alias url https://workers.mycloudflaredomain.com/?asn=24424&ipv4=yes&details=yes [http_code:500]
I am/was watching this particular request come in and the server returning a 200 OK response, however in the OPNSense log it says 500 and it does not import the values.
I am not a 100% certain what's going on here and I am not 'blaming' OPNsense here, I must be missing something.
Has anyone seen something similar before and knows what to do? I can gladly provide the full URL tables list for debug purposes, if needed.
Thanks!
-Joerg
Hi
Quoteworkers.mycloudflaredomain.com
cloudflare with load balancing?
^ +1
Can you curl the url from a console?
@benyamin: Yep curl requests on the OPNSense itself work just fine
and @Fright: yep.. but also tried several http://asn.blawk.net/ based ones and then went ahead and even started a local IIS instance on my desktop serving the responses / files
@jbattermann
any clue in IIS logs?
Quoteyep..
cloudflare may have some bot-protection. need special library for this imho
Quotetried several http://asn.blawk.net/
tried http://asn.blawk.net/24424 - worked fine for me