The misadventures of an SPF record

TL;DR: I ran a scan against the three million most visited domains and discovered that the Ukrainian Ministry of Defence, <REDACTED> University, University of Miami, along with 1000+ other domains had mistakenly used the “+all” SPF mechanism at the end of their respective SPF records – effectively meaning any public IP address can send SPF authenticated emails on their behalf. These results were validated through emails I sent to myself from a select number of the affected domains.

It’s become a pastime of mine to run periodic scans against millions of domains to discover hidden gems in SPF records (I know what you’re thinking – what a lovely hobby). Sometimes these gems come in the form of a dangling IP I can snag, other times they come in the form of a severe misconfiguration.

It’s been about 6 months since I ran my last experiment, so I decided to try something new. This time, I would scan the three million most visited domains to see if I could gather statistics around SPF implementation status’ and so on.

Now I won’t bore you with the details, but there are approximately 10 different checks I run when scanning a domain – ranging from Low to Severe in impact.

What is SPF? The Senders Policy Framework (SPF) is a public record that organisations publish on their Domain Name Server (DNS) and it contains a list of IP addresses that tell receiving organisations where they can expect legitimate and authorised emails to originate from.

Scanning Three Million Domains

To kick-start the scan, I collected three million domains from an amazing GitHub project called domains that’s maintained by a Researcher out of Ukraine.

Next, I had the challenge of performing the actual scan. Now the main hurdle here is time. To extract the full email sender supply chain (i.e., SPF record) for a domain, it takes about 6 DNS requests on average. This is because I’m recursively querying SPF sub-lookups and looking at DMARC records to check for any mitigating controls.

Performing a scan like this synchronously just isn’t feasible, I’d be waiting for months before getting the results! So, I decided to use AWS Lambda to scale from one to hundreds! I put together a script to run 400 concurrent Lambda functions in 2.5-minute intervals, with each scanning 15 domains and saving the results into a DynamoDB database.

What is AWS Lambda? It's a type of cloud compute offered by Amazon Web Services (AWS) that runs code in a highly efficient and horizontally scalable manner (i.e., the same piece of code can run hundreds of times simultaneously) - commonly referred to as serverless computing.

Using this method, I had the results in about 18 hours!

Trawling through the results

Like I mentioned, I’m scanning for a variety of SPF misconfigurations, so naturally I start by looking at those with a Severe impact :)

Just to clarify, when I say Severe impact, I mean implementation of the “+all” mechanism at the end of an SPF record. Use of the “+all” mechanism effectively broadcasts to the world that ANY public IP address can send SPF authenticated emails on behalf of the domain. This is leaps and bounds worse than every other type of misconfiguration or even use of dangling/unused IPs, so it always takes me aback when I see it.

Now anyway, the results… I extracted the data from my database, filtered for use of the “+all” mechanism and was presented with a list of 1,132 domains.

My next step was to filter down and see if there were any organisations of actual importance in this, so I added a “.gov” followed by a “.edu” filter and that’s when I noticed something surprising…

Many of these organisations are highly respected and in positions of trust. At this point, I realised the need to validate that these issues were real. So, I sent myself several test emails, appearing to come from some of the affected organisations that had used the “+all” SPF mechanism.

And as suspected, the emails were passing all SPF + DMARC checks, flying directly into my inbox... and the only issue? The use of a “+” instead of a “-“ at the end of their respective SPF records.

Something still bugged me

Note: In some instances, such as that of mil.gov.ua, I found that email providers would reject the SMTP connection when masquarading as the domain. I believe this could be the result of widespread abuse, which has resulted in email providers distrusting the domain in its entirety.

What's the impact?

Each of the 1,132 organisations that have mistakenly or deliberately used the “+all” SPF mechanism are highly susceptible to phishing-related attacks as anyone with an email server and a public IP address can send SPF authenticated emails on behalf of these domains. This risk also extends to their downstream vendors, customers, partners and even the general public.

Wrapping up

Maintaining control over your email supply chain, whether it’s your SPF configuration or the IPs listed within it, is absolutely crucial to ensuring your organisation and its customers, vendors, partners and even the general public aren’t introduced to unnecessary phishing related risks. If you’re unsure whether your organisation is affected, simply use the free domain scanning tool provided by CanIPhish.

Using this, you can visualise the full extent of your email sender and receiver supply chains, along with any issues associated to them. Finally, if you’re interested in hearing about the lower severity statistics associated to this research, stay tuned! I’ll look to publish these in the next fortnight.

Note: The organisations identified in this blog post have had the vulnerability responsibly disclosed either through direct contact or through report to their respective national CERT teams (e.g., USA CISA and Ukrainian CERT). A short remediation period was provided prior to the blog going live. The reason for this short disclosure period is due to the high likelihood that these issues are currently being exploited in-the-wild.