This week’s GoDaddy outage took down millions of websites – but it also took down thousands of VoIP phones. Many people aren’t aware of how their VoIP phone is programmed, but it must know how to reach the VoIP service provider and usually, this is done by a DNS setting in the phone. If your provider’s DNS was hosted with GoDaddy, your phone would no longer know how to reach the service provider and would stop working.
Our services remained up and active, mostly because we weren’t hosted at GoDaddy. :-) But even if we had been, our services would still have been OK thanks to an interesting strategy on DNS mitigation for VoIP Service Providers. Everyone can use this strategy, but most VoIP providers do not.
Our strategy stems from reading about past VoIP providers failures – namely a massive outage at Packet8 (8x8) in their early days. You can read more about the outage here and here.
How Things Are Today
VoIP, like most services on the web, is a weakest-link service. Any item between your phone and the final destination that is not operating properly and does not have a backup or failover system will cause the entire service to fail. Most people concentrate on making everything EXCEPT DNS redundant, which is a flaw in most design strategies.
As you can see in the left side of the diagram above, the phone and server portions of the network are redundant but the DNS server is not – there’s only one of them.
Many people are also unaware that their phone is smart-enough to be registered or proxy traffic through two different servers (some phones, even more!). You can use this to your advantage to create a redundant service that relies on two completely independent DNS providers or services.
DNS and Redundant VoIP Services
The strategy for redundancy is simple: on the phones themselves, define two proxy servers who are run by two different DNS providers. To do this, you’ll need two top-level domain names such as voip.my-provider.com and voip.my-provider.net. You can point the actual IP addresses at the same network address if you want, but preferably you would also point each domain name at alternate networks or, even better, use SRV to point each domain name at both networks. In this way you’ll end up with an architecture that looks more like the right side of the image below:
SRV records are the best strategy because you can have the phone or endpoint device still register to more then one server even when DNS is down, to avoid from a double-failure.
Alternate Strategies
Another simple strategy is not to rely on DNS at all. This can be dangerous if you have customers who provision their own phones and, later on, you move data centers or change IP blocks for some reason and have no way to change the IPs of those phones. Unless you own your IPs (via the ARIN registry), this can be dangerous. Consider this solution with care.
Learn More
Our wiki is always being updated with more information regarding DNS strategies and the like as they apply to VOIP so stay tuned there!