Note: This is part of a series covering the basics of email authentication. See the rest of the series:
SPF allows the owners of domain names to create rule sets that define whether an email server at a given IP address is allowed to send email on behalf of that domain or not.
In a simplistic sense, SPF lets you create a whitelist for IP addresses. If a mail server with an IP address that’s not on your list tries to send email using your domain, it won’t pass the SPF authentication test.
Work on SPF started in 2003, in an effort to control spam, and domains have been deploying it increasingly widely since then. SPF was published as RFC 4408 in 2006, and became an officially proposed Internet standard via RFC 7208 in 2014.
While the standard hasn’t been officially approved by the IETF, it is a de facto standard because it’s so widely used by major and minor receivers of email (Google, Microsoft, Yahoo, etc.) as well as all Secure Email Gateways (SEGs).
The way SPF works is quite simple in principle:
- Domain owners publish SPF records to the Domain Name System (DNS) that spell out the rule sets for their domains. An SPF record is plain text, and it can be as simple as a single line listing the IP addresses that are allowed to send email on the domain’s behalf.
- When an email server receives an incoming email, it examines the domain shown in the message’s Return-Path header.
- Using DNS, it checks to see if there is an SPF record for that domain.
- If there is a record, the receiving server then checks the IP address of the mail server that sent the message to figure out if it matches the SPF rules.
- If there’s a match, the email passes the test and, in most cases, is delivered to the user’s inbox; if not, the receiving server will typically reject the message or add a flag to it mark it as suspicious.
While simple in theory, SPF can be complex to implement in practice. One potential gotcha is that SPF records are text, but the syntax is a little tricky and it can be easy to make typos or other errors that are hard to spot and which render the SPF record useless. For instance, we examined the SPF records for all 62 sponsors of the 2017 RSA Conference. We found 58 who had published SPF records, but 17 of those had records with errors in them. That’s a nearly 30 percent failure rate — and that’s among security companies. Companies that don’t have a lot of expertise in cybersecurity in general (and email security in particular) often find SPF even more tricky. Even some companies that provide email security systems have issues with their SPF records!
Second, the rules (“directives” in the language of SPF) can be quite a bit more complicated than lists of IP addresses. For example, an SPF record can include:
- A list of specific permitted IP addresses or net blocks (ranges of IPs)
- Rules that point to other types of DNS records. For instance, “if the MX record or the A record for the sending server contains a specific IP address, let the message pass.”
- “Include” directives that reference SPF rules controlled by another entity.
For example, if you are using Google’s G Suite and you want to enable SPF for the mail sent by Gmail (as well as email notifications sent by Google Docs, Google Calendar, and so on) you would add “include:_spf.google.com” to your SPF record. This tells mail servers to do an additional DNS lookup for _spf.google.com to find additional SPF rules listed there. Many other cloud services that send email on their customers’ behalf — and there are thousands of them — use a similar approach.
Here are a few more potential sticking points with SPF.
Return-Path vs. From Addresses
The first big issue with SPF is that it uses the domain shown in a message’s Return-Path field for authentication, not the From address that humans can actually read. (To be fair, humans can read the Return-Path address too, but usually only by selecting an option to view a message’s full headers or the raw text of the full message. It’s not usually shown by default.)
The Return-Path is used to indicate where bounce messages (such as non-delivery reports) should go. In some cases, that address will have the same domain as what’s shown in the From field, but the domains may differ in other cases. For example, if you’re sending mail through a mailing list, the From field might show your address, while the Return-Path field shows the address of the email list. Or if you’re using a bulk mailing service like MailChimp, the Return-Path address might be an address that directs bounces to MailChimp, while the human readable From field shows your company’s address.
If the domain shown in the Return-Path for your messages doesn’t authenticate properly, then your messages may get rejected.
Worse, phishers can use the general invisibility of Return-Path to set up their own domains with their own SPF records. Then they can send out phish emails that appear to come from a trusted company or brand, with that company’s domain showing in the From field but the phisher’s own domain in Return-Path. Such messages are fake, but they will pass SPF authentication.
This is one reason why SPF is not a complete email authentication solution on its own. An additional standard (DMARC) addresses this issue by enabling the domain owner to require “alignment,” which means that a message must pass SPF and its Return-Path and From addresses must be the same. We’ll have more on that in a later post.
The 10-Domain Lookup Limit
The second big issue is that SPF contains a limit on the number of DNS lookups that mail servers will do when evaluating an SPF record. That limit is 10, and it was put in place because the creators of SPF were concerned about preventing denial-of-service attacks.
For instance, when an SPF record has an “include” directive that specifies a domain, the authenticating server needs to do a DNS lookup for that domain, in order to retrieve its SPF record, containing additional rules. While 10 lookups might sound like a lot, you can use them up fast, particularly because one lookup might contain other lookups of its own. For instance, “include _spf.google.com” actually comprises four different lookups, because the SPF record at _spf.google.com contains three more lookups of its own.
It gets worse, because many companies are now using a wide variety of cloud services, similar to G Suite, that send emails on their behalf (for notifications, customer emails, etc.). Every one of these, from Salesforce.com to Sendgrid, needs to be spelled out in an SPF record “include” directive if you want their messages to authenticate as legitimate emails from your company. Any of their includes may include multiple DNS lookups.
Even if you take care to count the total number of lookups, one of the vendors you use might change their own SPF record, changing the number of lookups they do, which then affects your total — and you’ll get no notification about that.
Receiving email servers don’t tell you if they exceed the 10-lookup limit when trying to authenticate an email. After the 10th lookup they just stop, even if there are more “includes” to evaluate. As a result, emails that the domain owner intends to authenticate may fail SPF, since the server may never get to the rule that would have authenticated them.
There’s no limit on the number of IP addresses you can include in an SPF record, so some organizations try to get around the 10-domain lookup limit by listing authorized servers by IP address instead of domain name. This is called “flattening,” but it has problems of its own, which we’ll address in a future post.
A third issue with SPF is that it doesn’t support email forwarding. For instance, if you’re an alumnus of a college that offers you a lifetime email address (@alumni.college.edu or something similar), that probably forwards email to your actual address (@gmail.com or your company address). Any SPF records from senders trying to reach you through that address won’t validate, because to the Gmail server, it looks like the message is coming from the college’s email server, not the source. The same problem applies to e-mail lists.
Automation Addresses SPF Limitations
SPF is an essential part of email authentication, but it wasn’t designed for the cloud era and is unwieldy if you use more than two or three cloud services that send email on your behalf. Hand-crafting SPF records and maintaining them in conventional DNS doesn’t work for most organizations. What’s more, SPF doesn’t ensure that mail gets delivered (if it passes) or that it gets quarantined/rejected (if it fails). In the absence of a DMARC record, how receiving servers handle a message that fails SPF is entirely up to them.
Valimail has automated the key aspects of email authentication, including SPF. Valimail’s Email Anti-Impersonation System includes a purpose-built DNS service that, via a one-time delegation, responds to queries for a domain’s SPF record. The Valimail system auto-generates a perfect SPF record dynamically for each query, eliminating the need to flatten SPF records (by listing IP addresses instead of domain names) and avoiding the 10-lookup limit, no matter how many services you authorize. And our service makes it easy to add or remove authorized senders with a single click. To find out more, request a demo today.
Continue learning about email authentication's key standards with our next article: What is DKIM?