OPNsense Forum

Archive => 23.1 Legacy Series => Topic started by: joecool on February 06, 2023, 09:42:48 pm

Title: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: joecool on February 06, 2023, 09:42:48 pm
I am currently on v23.1_6 using the legacy dynamic dns client and updating my domain on porkbun.com.  It doesn't have "normal" DDNS capabilities but does have an API (https://kb.porkbun.com/article/190-getting-started-with-the-porkbun-api).  They have a couple of clients linked there already but I'm using a homemade php script.  To call it, I am sending:

    Update URL: http://10.1.2.12/porkbun_ddns.php?domain=mydomain.com&hosts=*,ipv4&ip=%IP%
    Result Match: I have "SUCCESS" coming back from my update script if it succeeds.

I just noticed the message "Please make sure to upgrade to os-ddclient before 23.7 is released as this plugin will be removed from our repository". I installed the new plugin but it doesn't look like calling a custom URL is supported? Am I missing something and is there a way to support this setup?

If not, is there a way to beg the devs to add Porkbun as an option prior to v23.7's release?  As of right now, if the legacy DDNS client gets turned off, I'm hosed.

TIA!
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: HunvHunv on February 13, 2023, 11:11:49 pm
Hi,
I have the same problem.
Is there any solution? Would be a big problem for me, because I cannot change the provider for several reasons.
In my case in the legacy dynamic dns client I can call for the Update URL:
https://update.twodns.de/update?hostname=myserver.dynvpn.de&ip=%IP%
I don't see that I can do this in the new one.
If I select "custom", I get a "Server" field, but that field only accepts servernames/IPs and not Update URLs. So
1. What is this field for
2. How to call Update URLs in the new DDNS plugin?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 08:37:08 am
Hi,
I have the same problem.
Is there any solution? Would be a big problem for me, because I cannot change the provider for several reasons.
In my case in the legacy dynamic dns client I can call for the Update URL:
https://update.twodns.de/update?hostname=myserver.dynvpn.de&ip=%IP%
I don't see that I can do this in the new one.
If I select "custom", I get a "Server" field, but that field only accepts servernames/IPs and not Update URLs. So
1. What is this field for
2. How to call Update URLs in the new DDNS plugin?

@1. Try: update.twodns.de
@2. Apparently no way to enter them directly, but the plugin will generate something along the line of your URL.

You don't need user name and password for updating? Really? Then leave these empty. Try DYNDNS2 and if that doesn't work try DYNDNS1 as "Method". Try with TLS, or maybe without.

PS:

According to this here

https://www.twodns.de/en/faqs#faq2

I would enter your email as user name and your password in the plugin...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: HunvHunv on February 14, 2023, 09:11:12 am
Hi,
I have the same problem.
Is there any solution? Would be a big problem for me, because I cannot change the provider for several reasons.
In my case in the legacy dynamic dns client I can call for the Update URL:
https://update.twodns.de/update?hostname=myserver.dynvpn.de&ip=%IP%
I don't see that I can do this in the new one.
If I select "custom", I get a "Server" field, but that field only accepts servernames/IPs and not Update URLs. So
1. What is this field for
2. How to call Update URLs in the new DDNS plugin?

@1. Try: update.twodns.de
@2. Apparently no way to enter them directly, but the plugin will generate something along the line of your URL.

You don't need user name and password for updating? Really? Then leave these empty. Try DYNDNS2 and if that doesn't work try DYNDNS1 as "Method". Try with TLS, or maybe without.

PS:

According to this here

https://www.twodns.de/en/faqs#faq2

I would enter your email as user name and your password in the plugin...

Yes, I also need User/Password, but this fields are available in old and new Plugin.
It is working with dyndns1 and dyndns2 without "Force SSL" and when changing the backend from "ddclient" to "OPNsense".
I'm not sure if this is as intuitive and flexible as the old plugin... I think definitely not.
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 09:50:56 am
Hi,
I have the same problem.
Is there any solution? Would be a big problem for me, because I cannot change the provider for several reasons.
In my case in the legacy dynamic dns client I can call for the Update URL:
https://update.twodns.de/update?hostname=myserver.dynvpn.de&ip=%IP%
I don't see that I can do this in the new one.
If I select "custom", I get a "Server" field, but that field only accepts servernames/IPs and not Update URLs. So
1. What is this field for
2. How to call Update URLs in the new DDNS plugin?

@1. Try: update.twodns.de
@2. Apparently no way to enter them directly, but the plugin will generate something along the line of your URL.

You don't need user name and password for updating? Really? Then leave these empty. Try DYNDNS2 and if that doesn't work try DYNDNS1 as "Method". Try with TLS, or maybe without.

PS:

According to this here

https://www.twodns.de/en/faqs#faq2

I would enter your email as user name and your password in the plugin...

Yes, I also need User/Password, but this fields are available in old and new Plugin.
It is working with dyndns1 and dyndns2 without "Force SSL" and when changing the backend from "ddclient" to "OPNsense".
I'm not sure if this is as intuitive and flexible as the old plugin... I think definitely not.

What do you mean: "changing the backend from "ddclient" to "OPNsense"?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 14, 2023, 10:00:05 am
Since 23.1 you can avoid dealing with ddclient issues by using a OPNsense Python backend implementation instead. It implements Dyndns2 protocol (used by a number of providers) and Azure DNS currently.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 10:19:58 am
Since 23.1 you can avoid dealing with ddclient issues by using a OPNsense Python backend implementation instead. It implements Dyndns2 protocol (used by a number of providers) and Azure DNS currently.


Cheers,
Franco

So we have a third solution now and this is the way to go in the future? :-)
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 14, 2023, 11:09:27 am
Perhaps, but only if enough community resources go into providing further protocol support for the backend to be able to cover more providers deviating from the standard(s).


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 12:26:53 pm
... I use the ddns Plugin (new) in 22.7.last, works fine for two dyndns providers with Custom settings, so what would be needed to add these two to the drop-down menu?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 14, 2023, 12:28:42 pm
Which providers are these? From the looks of it the GUI just follows what ddclient thinks the underlying protocol is for the provider and your providers use a different one than dyndns2 then. That particular protocol would have to be added a Python code to bring the providers using it into the list. :)


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 12:36:00 pm
goip.de and ddnss.de, for both I use dyndns2.

"Server"s are www.goip.de and ddnss.de, just plain vanilla user name and password, "IP check method" is Interface at the moment.

"Force SSL" ist checked, I use ipv4 only...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 14, 2023, 12:46:30 pm
Perhaps these just work switching to OPNsense backend and using "dyndn2" provider then? They are current set as "custom", right?


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 14, 2023, 01:27:44 pm
Yepp, custom, both... Thought you were interested in adding them to the drop-down to increase ootb user experience and satisfaction :-D
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: xpendable on February 14, 2023, 04:58:12 pm
So does the os-ddclient plugin even need to be installed if using the OPNsense backend?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 20, 2023, 08:37:30 am
I had a look in the release notes

https://forum.opnsense.org/index.php?topic=32081.0

and documentation

https://docs.opnsense.org/manual/dynamic_dns.html

but can't find any details on the Python script mentioned by @Franco. Started to look in the GUI, finally found it in System -> Settings -> Cron (drop down...).

Apparently the new ddns plugin is not triggered by newwanip but rather spams the DynDNS provider with updates every 5 min (300 sec iirc). That was apparently not the case with the old plugin. For the Python script the default in the GUI is one update a day at 00:00.

The users of dyndns are now somewhere between a rock and a hrad place... :-(

Wouldn't if be possible to start the new Python script upon detection of a new WAN IP rather than at a fixed time interval? Just asking...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 20, 2023, 09:20:02 am
Apparently the new ddns plugin is not triggered by newwanip but rather spams the DynDNS provider with updates every 5 min (300 sec iirc). That was apparently not the case with the old plugin. For the Python script the default in the GUI is one update a day at 00:00.

It does what you set for "Interval" after all, although technically it says it checks for an update and only updates if one was found so I'm not sure what your analysis is based on saying it spams the provider, unless perhaps the provider does not accept the new IP in which case it needs to retry...


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 20, 2023, 10:17:00 am
I had a look in the log for the new plugin and saw many rejected update events this morning (with 23.1.1 latest), which I don't see with 22.7.last.

I can confirm the update attempts from the log of my DynDNS provider every 5 min saying "OK, no IP change" for the new ddns plugin in OPNsense with 23.1, but not for 22.7 OPNsense.
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 20, 2023, 11:09:46 am
Some people suggested the local cached stats might be wrong in 23.1 but I'm unsure why (PHP 8.1 issue?), which could produce this outcome.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 20, 2023, 11:46:39 am
The 5min updates are not only logged by the plugin, they are real and can be seen by the dyndns provider, see attached.

07:57:53 was the time of booting the box this morning, afterwards every 5 min update attempts, which are NOT present for the ddns plugin in 22.7.last, I repeat...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 20, 2023, 12:50:17 pm
I don't think you are wrong on that end, I just meant to correct that the code tries to cache the IP, but perhaps it fails so it ends up thinking it needs to update as per interval.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 20, 2023, 03:48:37 pm
Is a bug report needed against the latest version of ddns plugin, as the problem is not present for the older version (regression)?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on February 20, 2023, 03:59:52 pm
Yep, thanks.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on February 20, 2023, 04:28:38 pm
...I have no github account, where to start?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 20, 2023, 05:35:09 pm
Even on latest OPNsense 23.1.3_4 the DDns is broken at least for SPdyn, as the sense is trying to refresh IP every 5 min, although no change in IP. At SPdyn the updates are logged as successful, while sense still has errors in log file:

Code: [Select]
2023-03-20T17:20:26 Notice ddclient[8506] 14361 - [meta sequenceId="4"] FAILED: was not updated because protocol <undefined> is not supported.
2023-03-20T17:20:26 Notice ddclient[8506] 13171 - [meta sequenceId="3"] FAILED: updating : unexpected status (0)
2023-03-20T17:20:26 Notice ddclient[8506] 12339 - [meta sequenceId="2"] WARNING: updating : nochg: No update required; unnecessary attempts to change to the current address are considered abusive
2023-03-20T17:20:26 Notice ddclient[8506] 11446 - [meta sequenceId="1"] FAILED: updating zhnbgr.spdns.de: unexpected status (14)

every 5 min.... epic fail.
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: Patrick M. Hausen on March 20, 2023, 06:03:55 pm
...I have no github account, where to start?
1. Create a github account.
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 20, 2023, 07:10:41 pm
In my experience these bug trackers are piles of trash and another form of "write it to the wiki".

I'm not the only user observing this behaviour, why has nothing happend in the last month?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 20, 2023, 08:47:29 pm
I'm sure there are good and bad sides to GitHub. Let's take a look at a well-coordinated effort to fix recent StrongSwan regression in EAP-TLS:

https://github.com/opnsense/core/issues/6415

On the other hand, here is the ddclient maintainer on the issue that chemlud describes as "epic fail":

https://github.com/ddclient/ddclient/issues/481#issuecomment-1382166547

You need to know where to take your battles AND if you can actually win them.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 20, 2023, 08:48:34 pm
PS: https://github.com/ddclient/ddclient/issues/528
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 21, 2023, 10:14:54 am
Thanks for replying, franco, I switched the backend to OPNsense on "General Settings" and got

Code: [Select]
2023-03-21T10:07:15 Error ddclient Unable to read file /var/tmp/ddclient_opn.status
2023-03-21T10:07:05 Error ddclient Unable to read file /var/tmp/ddclient_opn.status

What's wrong here?

Is there no way to switch back ddclient to the version that worked well with 22.7.11 for the time being?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 21, 2023, 11:06:02 am
Looks like there wasn't any value in there before. Is that still the case?

# cat /var/tmp/ddclient_opn.status

It's not impossible to revert...

# opnsense-code plugins
# cd /usr/plugins && git checkout stable/22.7
# cd dns/ddclient && make upgrade


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 21, 2023, 01:47:11 pm
Code: [Select]
cat /var/tmp/ddclient_opn.status
is always empty, but adding a "." or "1" with nano doesn't help, after switching to OPNsense as backend it's always empty again (same error in log). Editing the file after switching to backend OPNsense results in no new log entries ("debug" level) and apparently triggers no IP updates to DYNDNS services.

Deleting

"/var/tmp/ddclient.cache"

(alone or in combination with /var/tmp/ddclient_opn.status) doesn't help either, as recommended here

https://github.com/opnsense/plugins/issues/3344

by sfdementor.

The commands to downgrade affects all plugins? I don't see any reference to ddclient in the commands?!?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 21, 2023, 02:08:56 pm
Sorry, to downgrade a plugin you need to select the proper one by changing the directory. I've fixed the previous post.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 21, 2023, 02:10:16 pm
I had a look at this ddns.cache file (while switched to ddclient):

The entry for spdyn shows no ip=... and mtime=0 while for other dyndns providers the correct IP and a non-zero mtime is shown.

But trying to edit the file with nano is not possible, everything is full of artefacts when trying to enter text. Is nano broken? or my serial console with minicom (2.8, opensuse tumbleweed)?

Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 21, 2023, 02:38:48 pm
... I switched the ddclient to 1.9_2 with the commands outlined above, hope that stops the 5 min interval update attempts at SPdyn...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 21, 2023, 04:44:04 pm
Is it possible to lock ddclient on a 22.7.11 before upgrading to 23.1 or will an upgrade remove all package locks?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 21, 2023, 07:10:44 pm
Locks are not possible through major upgrades because they could make the upgrade fail.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 22, 2023, 11:54:23 am
To sum up the current situation with dynDNS in OPNsense:

- Old plugin is "legacy" (although working) and will be discontiuned with 23.7.

- "New" plugin is discontinued by developer, no help in sight. Bugs since 23.1 spam some dyndns provider with unnecessary updates every 5 min.

- New OPNsense (python) script can't be started from GUI, currently.

This is not very positive overall, to say the least...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 22, 2023, 12:14:46 pm
I think that sums it up nicely. A lot of complains, no one willing to do better. This is how open source dies (in the case of DynDNS).


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 22, 2023, 12:31:35 pm
Again: I'm a user, not a coder...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on March 22, 2023, 12:41:35 pm
Right. I'm just trying to state the facts for the whole situation from user to software to developer.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: joecool on March 22, 2023, 02:33:37 pm
franco:  I have a php script that I use via the "Update URL" line in the legacy provider that handles PorkBun.  They use an API to do ddns instead of standard protocols.  Would this be helpful in any way?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 22, 2023, 03:10:05 pm
@franco, what would be a viable way forward to keep dynDNS functional in 23.7 and beyond?

No question there is a significant number of users vitally relying on this feature as can be seen from the plethora of posts and threads on this topic since the discontinuation of the "legacy" plugin has been announced. Let's start the discussion early, as 23.7 not far away...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: chemlud on March 24, 2023, 04:21:49 pm
So no discussion means.... no discussion?

Could we figure out an amount of money needed to be raised by the community to keep dynDNS updates via OPNsense functional 23.7 and beyond?

There is not that much time left to come up with a solution, I guess...
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: joecool on July 31, 2023, 04:30:38 pm
The time has come and 23.7 just got released but we aren't any closer here.  I looked online and pfsense just added Porkbun support here:  https://redmine.pfsense.org/projects/pfsense/repository/2/revisions/d9d91d5df28c235baba4bfe3cc7c792037df35c2

Is that something that could be ported over?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on July 31, 2023, 04:57:51 pm
> The time has come and 23.7 just got released but we aren't any closer here.

Not very nice... and clear progress has been made since March 24 when this was last posted. This is a warning to stay fair.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: joecool on July 31, 2023, 05:12:03 pm
I had no intention to be non-nice so sorry if it came off that way.  I use Opnsense and greatly, greatly appreciate all the hard work you guys do!

I meant that this chain had gone silent and I saw no indication that any changes had been made to support Porkbun's DDNS.  Is that not the case?  If so, can you point me to something to read?
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: franco on August 01, 2023, 05:27:11 pm
Maybe the new POST option can help implement custom services now. I expect this to be in 23.7.1.


Cheers,
Franco
Title: Re: Legacy DDNS retiring without a replacement for URL callbacks?
Post by: joecool on August 31, 2023, 05:15:21 pm
I upgraded to v23.7.3 and the ddclient POST option works fine.  I had to adjust my Porkbun script a bit to provide the expected responses, but otherwise it's a-ok!

Thanks guys!