Sender Policy Framework (SPF) is an open, DNS-based email authentication system that gives domain owners control over which IP addresses are allowed to deliver email on their behalf. Receiving mail servers check the SPF records for incoming messages, and if the delivering server has an IP address listed in the SPF rule set, the message is marked as authenticated.
It’s a seemingly straightforward standard that greatly improves email authentication, thus cutting down on spam and phishing attacks. It is easy enough to implement (or appears easy enough) that it is widely used.
However, SPF has some built-in limitations that trip up many people. In fact, it’s not so easy to implement it correctly. The biggest stumbling block is the domain lookup limit, which means that some senders are failing to authenticate with SPF, even though they should pass.
Also, SPF’s use of the Return-Path address (the primary way mail servers identify the source of email) without regard to the human-readable From address leaves a window open for phishing attacks that authenticate properly, but have a fake From address.
The SPF 10-lookup limit
As part of evaluating whether an email message passes SPF authentication, a receiving mail server may have to make one or more DNS lookups.
To protect receiving mail servers from denial of service attacks the standard includes a hard limit on the number of domain lookups such a server is permitted to make when evaluating whether an email message passes SPF. That limit is 10 lookups.
Detail from Emarketeer infographic.
Unfortunately, those lookups can add up fast. For example, Google asks its customers to include the SPF record _spf.google.com in their individual domain SPF records. Looking up the contents of this record in DNS we find that it includes three google.com subdomains.
So a Google customer whose SPF includes the Google-recommended record will actually have a domain lookup count of four — one for the _spf.google.com lookup, and one for each of the three Google subdomains referenced by the first lookup.
Now imagine you want to include the records provided by a number of other email service providers — each of which may have several DNS lookups themselves. The total domain count of the resulting record is just the sum of the individual contributions.
This limit was not a problem when most companies operated their own mail servers. But now that companies utilize third-party senders, and each one takes up 3 or 4 servers, you can max out the 10 lookups pretty quickly.
For example, your company may use Google Apps for Business, Workday, and Zendesk. All of those cloud services include the ability to send email on your behalf, which means you’ll need to add corresponding SPF records for each one. And each of those records may include multiple DNS lookups of their own.
Making matters worse, it’s not obvious when the SPF 10-lookup limit has been exceeded. SPF records consist of a set of rules, each one evaluated sequentially. So if a message is validated by one of the rules defined early in the SPF record, the message will authenticate even though the SPF record as a whole is broken.
Messages which are intended to be authenticated by rules that appear later in the SPF record will fail, because the receiver will stop evaluating the record before it reaches those rules.
As a result, domain owners should try to limit the number of includes in the SPF records for their domains. And email service providers should not recommend or require that domain owners add an ‘include’ directive unless it is absolutely necessary.
Some domain owners try to address this process with domain “flattening”: They replace some of the domain names in the SPF record with the actual IP addresses or ranges for the corresponding servers. This gets around the the lookup limit, but it means that if these servers change IP addresses, someone in the organization has to manually update those IP addresses in the SPF records too. That’s a time-consuming and error-prone process — exactly the problem that DNS was designed to fix.
The human-readable From address
It says this comes from Hulu. But can you trust it?
SPF records apply to specific Return-Path domains, not the domain in the human-readable From address. Unfortunately, that introduces another limitation, because humans don’t generally pay attention to the Return-Path when reading emails. Instead, they look at the name and email address that appears in the “From” field in their email client or Web-based email display.
That’s where Domain-based Message Authentication, Reporting and Conformance (DMARC) comes in. It addresses this limitation in SPF by requiring a match (called “alignment”) between the address shown in the human-readable From field and the server authenticated by SPF.
In cases where SPF authenticates but the domain doesn’t match the From address, that SPF authentication will be discarded by DMARC, resulting in a non-authenticated email address.
In 2016, non-DMARC-authenticating emails will start to appear differently in Gmail and Outlook. For instance, the company logo or avatar that usually displays alongside a person’s name may be replaced with a big question mark, as a signal that the email was not authenticated properly.
But we’ll have more to say about that in a future post.
In the meantime, check out our DMARC FAQ for ESPs for more details on implementing SPF and DMARC.