DDNS: Is a DIY solution possible? Better?

Question

I'm trying to establish a personal mail/calendar server at my home (yes, I've heard that it's difficult, it's a lot of trouble, and so forth, but I'd still like to try). I have an ISP that doesn't offer static IP addresses, so it looks like some sort of Dynamic Domain Name Service (DDNS) is the solution.

However, I've been doing research, and I've found at least a couple of online resources which explain that you can do DDNS yourself: you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using for your home servers (I happen to have a domain parked with a hosting provider for just this eventuality, and as I understand it, I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically...someone let me know if I'm wrong on this and there's a simpler way).

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address). However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

Also, I have another concern: are there any security issues I may be overlooking with having a DDNS provider? Won't they be able to monitor all the traffic flowing through the domain name they provide? Does anyone have an informed opinion with regard to which method (paid vs. DIY) might be better?

I appreciate your time...thanks!

Answer 1

I'm trying to establish a personal mail/calendar server at my home (yes, I've heard that it's difficult, it's a lot of trouble, and so forth, but I'd still like to try).

You probably won't have much luck with the mail part. See @Alex's answer.

you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using for your home servers

Pretty much that.

I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically

Yes, although if the company just provides a generic "host everything" service it might not have any DNS management API at all (focusing instead on web and mail) and you might need to move the domain elsewhere.

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address).

No. Only your DNS hosting provider's own systems have to be updated. The rest of the world doesn't keep a permanent record – it merely caches the results of individual lookups, for the duration indicated in each (sub)domain's "TTL" (Time To Live) field.

However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

I would guess that they allow configuring a very low TTL on the dynamic domains (down to a few seconds), which means it'll drop out of any caches very quickly, at the cost of the DDNS provider itself receiving many more requests (higher load on their DNS servers and databases, and an excuse to charge you more). That alone isn't something special and could be implemented with any DIY method.

Won't they be able to monitor all the traffic flowing through the domain name they provide?

No. The DNS server only provides you with an address (much like a phone book) and is not involved in any further communication.

(Unless the provider actually attempts to return false data, which would considerably shorten the company's TTL the moment news websites learn about it.)

That said, do pay attention at how the API works; of course you can't be sure that the service doesn't have any vulnerabilities, but if (e.g.) the API runs over unencrypted HTTP and transmits the API key in plain sight, then that's not something you want to rely on.

Answer 2

If you don't have static IP then you should forget about mail server if you going with DDNS solution, most email servers either would reject emails from you or will tag email with highest spam level since all dynamic IPs are in PBL lists. (You can see more details in P.S. section why it isn't a good idea to have emails server on residential IP, but there are still a workaround way by using intermediate cheap VPS(virtual private server))

Regarding "DDNS yourself" - a good domain registrars providing for free IP updating via their API, all your program need to do is to verify periodically public IP and if it changed send new IP to registrar who will update A(AAAA) record. BTW, most nowadays routers already have such feature (watch for IP and report to DDNS provider)

I've read that it may take several hours to propagate throughout the system/world

It depend on DNS provider, respectful registrars allows to set TTL (time that tell others how often IP might be changed) equal to 5 minutes. Not all forwarding intermediate DNS servers following this to avoid high loading, but usually even if they won't follow domain owner TTL, that rarely go longer than a few hours. Most of forwarders will update their caches as you would set in domain TTL.

are there any security issues I may be overlooking with having a DDNS provider?

Going online it is already possible security issue. Isolate your server from local network to avoid unwelcome guests.

Does anyone have an informed opinion with regard to which method (paid vs. DIY) might be better?

You will throw your time and money to the air if you would go with DDNS. Nowadays you can get decent VPS (virtual private server) for 3-4 bucks per month. While web site (if you plan to have one) can be hosted directly on VPS since usually it doesn't take a lot of space, email server could be problematic if you expect to run your server for a long time or expecting high volume of emails. Usually 20GB space enough for small businesses up to 3-5 years even without deleting old emails. Even if you expecting huge amount of emails, you can use nginx feature to proxy email traffic to your home. So you can host primary email server at home on dynamic IP and VPS (that has static IP) will proxy incoming/outgoing traffic to your home. You can use your own VPS in such configuration without pain since there no need to worry about DNS propagation, domain will be always pointing to static IP of VPS. You still need to manage reporting your home IP changes to VPS, so VPS would know where to proxy traffic, but it is much easier, just query some URL on your VPS and parse in logs your incoming IP and adjust nginx, so it always know where you are.

P.S.

---

I can see that this topic is interesting for superusers, so I would add a little more details.

PBL lists holds database of IPs that generally are dynamic IPs, so PBL helps a lot to operators of emails servers. It isn't a technical issue or ISP are a bad guys to not allow email server on dynamic IPs, the issue is that most email traffic from dynamic IPs are coming from infected computers that sending spam or malware at huge volume that can easily DDoS receiving server if one is a target. Some ISP blocking outgoing connections to port 25 to prevent spreading of malware and DDoS, but some doesn't. Practically all corporate email's server simply dropping connections that comes from PBL list that reduce spam significantly.

The second one effective antispam solution is to drop connections from IPs that doesn't have reverse PTR record in DNS and doesn't match domain's DNS record. Even if connections comes from static IP that haven't PTR record, it usually either a badly misconfigured setup or mostly it comes from servers running by spam gangs (there could exclusions for some big (but careless) providers, but they can be added manually in whitelist). While it is a matter of a few minutes to set reverse PTR record on VPS, it isn't a case if static IPs obtained from ISP and process to set PTR is usually a PITA (one have to call them, submit a ticket after verification that you original owner of IP and wait for mercy of their sysadmin who need to set reverse PTR record, sometimes in a few hours, but sometimes a days)

Also, not a critical, but... to avoid email forging, most email servers owners using so called SPF(sender policy framework) that allows to specify most fast policy processing method if one set in DNS authorized IP addresses that allowed to send emails on behalf of domain. (one can specify authorized servers by FQDN as reference to MX record, but it extra round trips over DNS for connecting servers) So, managing floating IP in DNS wouldn't be a fun.

Answer 3

I have an ISP that doesn't offer static IP addresses, so it looks like some sort of Dynamic Domain Name Service (DDNS) is the solution.

That's one solution. As an example of another solution, a HurricaneElectric.net IPv6 tunnel provides a static (IPv6) address with a movable tunnel endpoint. Granted, at this time, IPv4 would be nicer to support for such a functionality with the generality public, but if you can find a willing cooperative computer, you could technically do such a thing with IPv4 too.

you need to have a script/program which monitors your IP address periodically, and if the address changes, then the script/app needs to update whatever domain name you're using

This sounds like a technically solid plan.

I just need the API key of the hosting company in order to adjust the necessary domain/IP records programmatically...someone let me know if I'm wrong on this and there's a simpler way).

The exact details would depend on the domain name registrar's choice of how they implement this feature. Some may use an API key of some sort, while others might rely on a web interface for automatic updates. In the old days, some ISP's provided such a service, but relied on manual changes in response to requests. So it's entirely up to whoever provides you with the service.

Here's the thing: when you update your domain-name records in the fashion I've described above, I've read that it may take several hours to propagate throughout the system/world (all DNS servers have to be repopulated with your updated address).

Bah humbug. DNS propagation has been known to take minutes or hours or days (e.g., 72 hours). However, when people have heavily analyzed things, they've found that much of that vague "propagation" time was simply from a DNS hosting provider being slow to update.

In better theory, you should just need to wait for the TTL value. Although, there is a problem with that theory...

However, several paid DDNS providers I've been looking at seem to promote their ability to have the change take effect near instantaneously (or at least, faster than my DIY-method). Is that true? Is there something I've missed?

Okay, here's the reality: For your update to take full effect, you'll need to have the Internet flush its active cache of old information.

According to the standards, caching DNS servers may rely on their cache for the length of time specified by a TTL value that you can configure.

However, the reality is that at least some (and maybe even most?) very large ISPs have been known to run their own caching DNS servers which have been known to just completely ignore the TTL values. They do this because they feel like if they update their DNS caches less frequently, the overall effect will be less bandwidth (and maybe some less computing time).

So, any E-Mail server that relies on such a DNS server may be affected, and be unable to notice your updates until the DNS server gets updated. In some cases, that may take a day or two (or three?).

However, such effects have become increasingly rare. In actual practice, most DNS servers will have their caches flushed within an hour or two.

Since some caches won't be updated quite as quickly as some others, the effect is that some places on the Internet will work with the new address, while other places will still try to use the old address. Within a couple of hours, most computers will work just fine with the new information. (Many, many of them may work within minutes.)

The typical behavior of E-Mail software is to try to send the E-Mail. If that fails, then try again later. E-Mail servers will typically keep retrying (maybe about once an hour) for days before giving up. So what's likely to happen is that you won't lose E-Mail, but it will get delayed a bit.

Alex's comment "all dynamic IPs are in PBL lists" is clearly wrong, as this information is de-centralized (so the word "all" is inaccurate), but it is true that many dynamic IPs are in such lists, and so that may mean that some computers/devices related to E-Mail may decide to not cooperate with you.

Also, I have another concern: are there any security issues I may be overlooking with having a DDNS provider?

The biggest concern will be whether your updates are handled with a secure fashion.

Won't they be able to monitor all the traffic flowing through the domain name they provide?

No. The DNS server's job is to receive a request for a domain name, and provide a response. The traditional typical response is to provide one or more IP addresses. Other responses are possible, such as referring to another DNS server or domain name (e.g., with a CNAME), or other data (e.g., helping to provide security through the newer DNSSec standard).

Does anyone have an informed opinion ...

I'd like to point out that if you really want to run a serious E-Mail server, you may want to consider being compliant with modern E-Mail standards. That involves more than just being compliant with SMTP and DNS technical specifications. Many people use large providers, and those large providers may implement their own expectations.

For example, I know of an E-Mail server that was set up with Debian and Postgrey years ago. Postgrey is some software that provides "greylisting" anti-spam handling. However, the version of Postgrey that is used assumes that when an E-Mail server retries the E-Mail, the sending E-Mail server will use the same IP address when doing so. Office 365 E-Mail servers have been known to retry sending an E-Mail from a different IP address that is still within an IPv6 /64 subnet. Postgrey doesn't like that.

As more and more organizations have switched to Office 365, this has become more and more of a problem for people using that old E-Mail server. A newer version of the Postgrey software has been released, but the easy way to install such software is to use the official software repository for that operating system. So, in practice, the smart way to update that software will be to upgrade the operating system.

There are other conventions, such as having DNS names that start with "mail." which can cause your setup to be judged as being more or less trustworthy. This may impact whether devices treat you like a non-compliant spammer, or like a device that is worth communicating with.

Sure, maybe when speaking very strictly about official technical specifications, giant organizations are performing some actions that are different than minimum requirements demanded by the RFC documents that contain the technical specifications of the protocols being used. But if you want to communicate with the larger Internet community, there are some additional standards that get imposed by some significant/large players. Be prepared to meet those standards well, or be prepared to encounter some troubles.

I'm being a bit vague about exactly what all those standards are, because they can change over time.

Regarding that old E-Mail server that will need to upgrade its old Debian operating system, maybe people should be upgrading their operating system more frequently anyway. The point I'm making, though, is that a software setup that worked perfectly well for years is now broken, because of newer behavior that is commonly being used by many E-Mail addresses. If you try to do unusual things, like using Dynamic DNS on a slower Internet provider, you may be more likely to encounter some extra problems along the way. As you sound ambitious, maybe you can invest the efforts into that. I'm just warning you to prepare to need to do that.

... with regard to which method (paid vs. DIY) might be better?

As others have pointed out, paid will be much easier, and is pretty economical for most people. Large provides are likely to provide a stable IP address that you can have your MX record point to (so E-Mail goes there), and may provide notably better bandwidth.

DIY is better for gaining experience and learning how things work, and choosing not to rely solely on implementations from major corporations. Having more control over your implementation can also allow you to make significant custom changes much more rapidly.

Which is "better" will depend on your individual goals, so I leave such conclusions up to you.

Answer 4

Yes it's possible host the DDNS server yourself (in fact, I've written one DynDNS compliant implementation for DJBs tinydns myself some time ago). You can use regular DynDNS client on dynamic IP to update your DNS name, but the server code would still to run somewhere with static IP (as the GTLD servers still need to point NS records for your domain to some static IP).

Also, if you don't host your DNS yourself but offload that to third party DNS provider, then they might be giving you the API to changing IP as you say. It would probably propagate slower than proper DDNS solution, and you would obviously not be fully self-hosted but depend on that third party for DNS services. They are usually not malicious (and there are so many other security issues on the internet to be worried more about), but I would still prefer solution under my control (if nothing else, as it gives me freedom to choose my DNS provider instead of being locked in).

I also do run my own Calendar/Contacts/Pictures server instances instead of handing over data to google or someone else (by using open source Nextcloud and Mediagoblin instances). While I run them on "my" VPS with static IPs (as it is both easier and provides more bandwidth), both could be run on dynamic IP you have at home.

Also, as others mentioned, for sending mail you won't be able to reliably send mail directly from that server due to PBL blacklists (it will work sometimes, be lost or bounced other times, or marked as SPAM), but if you're OK via forwarding it via some other SMTP server (like from gmail or your ISP) it will work just fine.

However, the incoming mail (MX) server on dynamic IP would be is pretty problematic. For example, if mail happens to arrive while you're changing IPs it's possible it will try to deliver to IP you had some moments ago, but now some other customer has. In most cases, it would simply result in failed connect and thus delayed mail, but if other customer also runs SMTP server the mail would be delivered to him/her or be bounced back with "recipient does not exist" fatal error (which will usually automatically unsubscribe your from mailing lists etc). Also, if your IP doesn't have reverse DNS (or sometimes even it has but if it looks dynamic!), some mail servers will refuse to deliver you mail.

But if some percentage of lost or misdirected mail is not a problem for you, then by all means feel free to try it.

Note that is is also possible that your ISP have rules against you running services on your low-priced "consumer internet", and might shut you down if they find you breaking the contract. Or they might even be actively blocking any incoming connections to your home on lownumbered popular ports like tcp/25, 80, 443, etc. which could destroy your whole idea of self hosting (unless you change ISP or go in more expensive "bussiness class" which also usually has static IPs).

And last thing to note is that in many typical consumer home internet connections (like ADSL, cable, etc) the speed is asymmetric - that is, your download is much faster than your upload speed. Which is bad if you want to serve some bigger public content like pictures (instead of relying on facebook or google for that and giving up your rights) from your home, as it will be slow (not also for the guy downloading pictures from you, but also for your whole internet connection).

Answer 5

Other answers have already explained DDNS part.

I am going to explain why you have to use a separate server to send email (since brief explanation from @Alex is incomplete).

Most importantly, you need a valid reverse PTR record to send email — many email servers will check it and bounce your mail, if reverse DNS record for IP address does not match sender domain. This record is provided by the owner of IP address — e.g. by your ISP.

Now let's imagine, that you have somehow gotten valid, dynamically updating reverse DNS (ha-ha). You still have to convince everyone, that your domain is legit, and your outgoing email isn't spam.

As explained by @Alex, small-time mail hosters love to use spamhaus and other online blacklists. But I have seen those corporate admins do a lot of other dumb things (such as blocking all email, that does not come from Gmail/Hotmail). Actually, it is not just some "corporate admins" ­— I have seen Sourceforge block registration from legitimate corporate email domain, because "we don't know why, but our spam filter thinks, that you are bad". Just ignore them — you can't stay compatible with everyone under sky.

Huge mail hosters these days don't rely on spamhaus or other PBL. They track your trustworthiness themselves. The sender reputation (at least most of it) is attached to the IP. This is because spammers frequently get a boot from their hosters, so they are forced to jump IPs. From the viewpoint of Gmail your recently created domain/IP is no different from common spammer. When you start sending email, your reputation is low (you are treated as spammer by default!). Most of your outgoing email will be marked as spam. When someone replies to your email or especially marks it as legitimate (by pressing corresponding button in web-interface of their email provider), the trust towards you will slightly increase. As you can see, to grow sender reputation, you'd have to use the same domain on the same IP over years. This can't be done reliably with dynamic IP.

---

Once you lease a VPS from hoster, keeping a home server on dynamic IP will become a lot easier. You will be able to use that VPS as your own DDNS server with extremely low TTL. You might even be able to forego DNS and use other means (such as HTTP redirection) to handle changing IP of your home box. You will still be able to receive email directly to your home box — optionally with fallback to the VPS, when your home IP is down or recently changed.

Answer 6

It can be done on CPanel using a PHP script - just ensure you set your security correctly (HTTPS, etc.) and read the ToS from your hosting provider.

https://www.shadowsplace.net/1231/internet/use-whm-cpanel-as-dynamic-dns-service/

If you mimic a popular dynamic DNS providers URL scheme, you can use the built-in Dynamic DNS functions in 'black-box' equipment (e.g. NAS, IP Cam, etc.) and get your router to redirect the request to your host instead.

Answer 7

Running your own public DDNS service is entirely possible. Propagation is not a concern as the only thing that must propagate globally are your name server settings. Any changes you make in your local DNS zones can be instant depending on the TTL you set for your records. There is concern for spam and bad players using your service and domain for phishing attacks etc.. That said there are some pre made applications that can accomplish this goal and provide anti spam utilities as well. I would recommend MintDNS for Windows servers https://dynamic.domains and GNUDIP for Linux servers http://gnudip2.sourceforge.net/. MintDNS is pretty widely used, and you will see it in action on the Q-See DDNS, Nightowl DDNS, Swann DDNS sites and thousands of others etc.. I have played with both, and MintDNS comes with scanning utilities, not unlike virus scanning utilities to help weed out bad players etc.. There is also no reason you could not field your own home brewed DDNS. I believe all use TSIG updates etc..

Answer 8

My humble additon to dyndns service, which is free and running on many systems (also on routers) - freedns.afraid.org

I'm using it to have my home IP address pointing to some selected domain name which the list offers free. My rPi executes "curl" script and it works good for me (well, since I'm not running mail server that is not a problem if 10 minutes is not available).

I had same idea for running mail server also at home, but ISP blocked traffic, so I had to abandon it :-(

Best Regards.