SPF Records and Too Many DNS Lookups
Some years ago, I added an SPF record to my domain. This makes it more likely that e-mails I send will get through to their recipients. The basic idea is that you declare (in your DNS record) which servers should be sending mail from your domain; any message not sent from one of those servers is probably spoofed. My SPF record for c-command.com looked something like this:
v=spf1 ip4:69.163.248.94 include:dreamhost.com include:fogcreek.com include:amazonses.com mx -all
The IP address was for my server. The SPF record also includes the SPF record for DreamHost, since I often send mail from their shared mail servers; Fog Creek, since I use FogBugz to send e-mails to customers; and Amazon SES, which I use to send order confirmation e-mails and serial number lookups. The mx means to look up the mail servers in my domain’s MX DNS record. The -all means that these are the only authorized servers.
It turns out that many mail servers don’t care if your SPF record is invalid. But some do, and I found this out when I was unable to send messages to a customer who had configured his server more strictly. When moving my site to a different server, I had updated the SPF record with the new IP address but accidentally included a space before it, which made the SPF record syntactically invalid.
I fixed this and, after waiting for the DNS to propagate, validated my SPF record. As expected, the syntax was now correct. However, the SPF record was still invalid:
Results - PermError SPF Permanent Error: Too many DNS lookups
I had not seen that before, but it turns out that RFC 7208 says:
Some mechanisms and modifiers (collectively, “terms”) cause DNS queries at the time of evaluation, and some do not. The following terms cause DNS queries: the “include”, “a”, “mx”, “ptr”, and “exists” mechanisms, and the “redirect” modifier. SPF implementations MUST limit the total number of those terms to 10 during SPF evaluation, to avoid unreasonable load on the DNS. If this limit is exceeded, the implementation MUST return “permerror”.
My SPF record looked like it only had four lookups:
$ host -t txt dreamhost.com | grep spf1 dreamhost.com descriptive text "v=spf1 ip4:62.229.62.0/24 ip4:69.64.144.0/20 ip4:98.124.192.0/18 ip4:66.33.206.0/24 ip4:66.33.195.34 ip4:208.113.189.254 ip4:208.113.200.0/24 ip4:66.33.216.0/24 ip4:208.97.187.128/25 ip4:64.90.62.0/24 ip4:64.90.63.0/25 ip4:64.90.63.128/26 include:_spf.goo" "gle.com include:sendgrid.net ~ALL"
$ host -t txt fogcreek.com | grep spf1 fogcreek.com descriptive text "v=spf1 ip4:64.34.80.172 ip4:69.90.190.164 include:_spf.google.com include:spf.mail.intercom.io include:amazonses.com -all"
$ host -t txt amazonses.com | grep spf1 amazonses.com descriptive text "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 -all"
$ host -t mx c-command.com c-command.com mail is handled by 0 mx1.sub5.homie.mail.dreamhost.com. c-command.com mail is handled by 0 mx2.sub5.homie.mail.dreamhost.com.
Unfortunately, the lookups within the dreamhost.com and fogcreek.com SPF records count, too. And those SPF records themselves have includes. The limit is quickly exceeded.
I was able to reduce the number of DNS lookups by removing include:dreamhost.com, because combination of my server’s IP address and the MX record were sufficient. DreamHost itself sends e-mails via Google and SendGrid, but it doesn’t do this on my behalf. Likewise, I could replace include:fogcreek.com with the IP addresses of the actual FogBugz mail servers, which are the only ones that FogCreek uses to send messages from my domain.
Now my SPF record looks like:
$ host -t txt c-command.com | grep spf1 c-command.com descriptive text "v=spf1 ip4:67.205.8.149 ip4:64.34.80.172 ip4:69.90.190.164 include:amazonses.com mx -all"
and it validates:
SPF record passed validation test with pySPF (Python SPF library)!
I will have to update it if any of the hard-coded IP addresses change, though.
2 Comments RSS · Twitter
Manually "flattening" SPF records might work at first, but if the hard-coded IP addresses change (which is reasonable enough), some automated SPF record flattening services go a long way.
For example, Dmarcly (https://dmarcly.com) has a feature called Safe SPF, which automates SPF flattening for you. It keeps checking the underlying IP addresses, and if there is any IP change, Dmarcly picks it up and updates the flattened record. This way, there is no more outdated IP address, and of course, one will never run into the too-many-dns-lookups issue. In addition, you don't have to do anything manually!
