Email authentication — and the trustworthiness of Internet email in general — took a huge leap forward this month with the IETF’s official publication of a new RFC.
RFC 8617, the Authenticated Received Chain (ARC) Protocol, describes a protocol for mail handlers to preserve authentication data when forwarding email messages.
This is important because certain types of legitimate mail can fail to authenticate when they’ve been forwarded before receipt, and therefore look suspicious. Mailing lists and SEGs frequently modify messages’ contents, exacerbating this problem. ARC allows these systems to continue forwarding email messages, without causing those messages to be treated as if they were spoofed.
The RFC means that ARC, which has been under development for several years, is no longer merely a theoretical solution to these problems. It’s real — and with the publication of an official RFC, it’s well on the way to becoming a full-fledged standard.
What’s more, there’s a world of open source software, including libraries in Python, Perl, and C, and support in the latest versions of the most widely deployed mailing list suites in the world, Mailman and Sympa. Google and Fastmail already have full ARC support, and other major mailbox providers are currently testing their implementations.
UPDATE November 2019: Microsoft has announced support for ARC within Office 365.
The long journey to RFC 8617
This outcome was far from certain. Every standard has many contributors and depends on the collaborative work of dozens of people and the institutional support of many companies. ARC is no exception to this. But despite years of effort, progress was sluggish at the time that Valimail became involved in it, almost 3 years ago.
Valimail identified ARC as a necessary solution to ecosystem problems that would create marked improvement for authenticating messages and close a key gap through which phish arrive in users’ inboxes. To help accelerate work on ARC, Valimail engaged on several fronts: first, we worked with the IETF to drive standards work forward; second, we contributed to and patronized open source software to support ARC; and third, we evangelized ARC through industry organizations and direct outreach.
Valimail began our ARC journey with the development of an ARC test suite that developers and engineers could use to make sure their implementations functioned to spec and interoperated correctly. We contributed to open source packages for implementing ARC in software (such as libraries in C and python, and mailing list software like Mailman). We also helped fund development of Perl libraries and support for the mailing list Sympa. And we rolled up our sleeves and took all the lessons learned from this development into the standards-making process, influenced the spec, updated the code, and repeated. Again and again. What that meant in practice was that I spent a lot of time in bug boards, on conference calls, and attending committee meetings in various locations around the world.
I’m honored to be listed as one of the authors of RFC 8617, recognizing Valimail’s contributions to this standard. But there’s no doubt that we couldn’t have done it alone. From Valimail, Peter Goldstein, Gene Shuman, and Mark Eissler were instrumental. And from the ecosystem at large, significant contributions from Murray Kucherawy, John Levine, and Scott Kitterman also made the difference in moving ARC from a dream into reality. A huge thanks also goes out to our colleagues at Google, especially Sri Somanchi, Brandon Long, and Neil Kumaran.
Why ARC matters
ARC addresses some edge cases that were impeding DMARC adoption (mailing lists, forwarders, and SEGs within enterprises). These account for just 1-2% of global mail volume, but that’s an incredibly large number of messages.
Here’s the problem: When messages pass through mailing lists, email gateways, and other message-modifying filters, they often fail to authenticate. Sender Policy Framework (SPF, RFC 7208) breaks under most forwarding circumstances, and DomainKeys Identified Mail (DKIM, RFC 6376) breaks when messages pass through forwarding services that modify content covered by the DKIM signature.
When a message loses its ability to be authenticated due to forwarding, and a policy is supposed to be applied to messages that fail to authenticate, as with Domain-based Message Authentication, Reporting, & Conformance (DMARC), these legitimate messages are treated as if they have been spoofed.
ARC solves these problems, by providing a means for these forwarding and filtering systems to attest to the authentication status of a message at the time they receive it. These attestations are then signed and bundled with the message as it is forwarded (calling “sealing”), creating an ARC Chain as multiple parties participate. Utilizing this chain, validators further along can examine the attestations to verify that the message was properly authenticated when it originated.
It’s as if you were on an international journey with many stops along the way. You present your passport at an airport in Hungary, and the customs officers inspect it and find it to be valid, so they stamp it with a visa showing that they approve your identification. When you get to Germany, the customs officers have never seen your country’s passport before, and they can’t independently verify its authenticity — but they have seen Hungarian visas and can tell that your Hungarian visa is valid. On the strength of the Hungarian endorsement, the German officials decide to approve your identification as valid, so they put their own stamp in the passport.
ARC works at scale
With ARC, the Internet mail ecosystem now has a way to preserve authentication information even across the most complex mail routing scenarios. This opens the door to more widespread adoption of authentication, which will help create a more secure ecosystem overall.
ARC works at scale, for both large- and small-volume senders and receivers. For large-volume mail receivers, reliable attribution solves an enormous problem about handling incoming mail. ARC allows authorization information to flow when internal routing is particularly complex, solving previously intractable authorization problems with multi-tenant, multi-party email routing. And since many large mail systems are really conglomerations of smaller mail systems bolted together, authentication information is frequently lost when mail moves between the subsystems. ARC simplifies the whole matter, and lets larger mail systems work as cohesive entities sharing authentication information properly.
For more information on ARC
ARC is not merely a protocol. There is a world of community support, and an ARC test suite (https://github.com/Valimail/arc_test_suite) that has proven useful in identifying specific issues and potential bugs to focus on at each interop. There are also numerous mature open source libraries that support ARC:
- C: https://github.com/trusteddomainproject/OpenARC
- Python: https://launchpad.net/dkimpy
- Perl: https://metacpan.org/pod/Mail::DKIM
And adoption of ARC has already begun. Google and Fastmail are currently using it publicly, and other operators are testing it internally, with public rollouts to come. Sympa already has ARC support, and Mailman3 merged in ARC support in mid-June 2019.
Building the future
Our contributions to ARC are not the end of Valimail’s community contributions, by a long shot. We are also deeply involved in the BIMI standard and DMARC 2.0, among other projects. Our approach in all of these is straightforward: We identity key gaps in the email ecosystem, then show up, collaborate with other industry partners, and make it a priority to provide resources toward building solutions. The results are improvements that help the entire ecosystem (and anyone with an inbox), whether they’re a Valimail customer or not.
In conclusion, we couldn’t be happier about RFC 8617, which will improve the security and trustworthiness of email-based communications, to the benefit of all Internet users. We are proud to have played a role in advancing ARC to this stage, and we are actively building a better Internet for the future.
If you’re interested in joining our team and contributing to ecosystem-scale progress like this, we’d like to hear from you. Find out more about the Valimail team here.