Reducing DNS lookups using SPF flattening

In SPF, a DNS lookup is the process using which the receiving mail server fetches the SPF TXT record of the sender’s domain. This is done to verify if the sender is authorized to send messages on behalf of the domain owner. However, to prevent abuse and overburdening, RFC has imposed a limit of a maximum of 10 DNS lookups per SPF record. Now, each mechanism triggers recursive DNS queries. This means that each instance of mechanisms like ‘include,’ ‘a,’ ‘mx,’ etc. counts towards this limit. Also, if a DNS query returns NXDOMAIN or empty, it’s also counted towards the limit.

If your SPF record exceeds the DNS lookup limit, it becomes invalid. In simpler words, it stops performing SPF authentication checks or triggers false positives. So, it’s obvious that you can’t afford to exceed this limit; otherwise, the whole purpose of deploying SPF will defy. So, what can you do to stay within the limit? Well, you can always use SPF flattening tools to reduce the number of DNS lookups. Let’s see how all of this works.

SPF and DNS lookup process

Upon receiving an email from your domain, the recipient’s server performs DNS lookups to retrieve the corresponding SPF record and verify if the message has been sent from an authorized server. The steps flow like this-

1. Extract the MAIL FROM domain

When an email is received, the MAIL FROM (Envelope From) domain (e.g., example.com) is noted. SPF checks are performed against this domain, not the visible ‘From’ field in the email.

2. Query DNS for SPF record

The recipient’s mail server performs a DNS TXT record lookup for example.com to get to know the SPF policy. 

3. Evaluate SPF record

The SPF record contains mechanisms like ‘ip4,’ ‘ip6,’ ‘a,’ ‘mx,’ ‘include,’ etc. Each may trigger further DNS lookups:

Mechanism	DNS lookup type	Description
ip4/ip6	None	Directly compares IP addresses; no DNS lookup needed.
a	A record lookup	Resolves the domain’s A record to get an IP address.
mx	MX record lookup	Retrieves the domain’s MX records, then resolves their A records.
include	TXT record lookup	Retrieves SPF record from another domain (recursive check).
exists	A record lookup	Checks if a domain resolves to any IP address.
redirect	TXT lookup	Redirects to another domain’s SPF record (like include).

4. Recursive DNS lookups

When the server encounters ‘include,’ ‘mx,’ or ‘a’ mechanisms, it performs additional DNS queries. These recursive lookups can result in multiple DNS queries.

5. Evaluate results against the sender’s IP

The IP address of the sending mail server is compared against the authorized IPs from the SPF record. Based on the match, SPF evaluation returns:

• PASS: Allowed

• FAIL: Rejected (-all)

• SOFTFAIL: Not recommended, but still accepted (~all)

• NEUTRAL: No policy (?all)

• PERMERROR: Invalid SPF record

• TEMPERROR: Temporary DNS issues

6. Decision to accept or reject

The result of the SPF check helps the recipient’s mail server decide whether to accept, flag, or reject the email. It is often used with DMARC to enforce email policies based on SPF and DKIM.

How SPF flattening helps reduce the number of DNS lookups

Staying within the lookup limit is difficult, especially if your email infrastructure has dynamic IPs and multiple servers. If this limit is exceeded, your SPF record will hit the Permerror, causing email delivery issues. 

Here’s an example of an unflattened SPF record–

v=spf1 include:_spf.google.com include:vendor.com include:vendor2.net -all

However, SPF flattening resolves all IP addresses upfront and replaces ‘include,’ ‘a,’ ‘mx,’ and other mechanisms with a direct list of IP addresses. 

Here’s an example of a resolved SPF record–

v=spf1 ip4:203.0.113.5 ip4:192.168.1.1 ip4:185.23.45.67 -all

In the above example, IPs have been resolved in advance, so there is no need for DNS lookups during the email validation process.

Let’s break down how exactly SPF flattening reduces DNS lookups-

SPF mechanism	DNS lookup type	Flattening behavior
include:	TXT lookup (and possibly others)	Resolves all IPs from the included domain upfront and replaces them with IP addresses.
a	A/AAAA lookup	Resolves the A/AAAA record and replaces it with IP addresses.
mx	MX lookup → A lookup	Resolves MX records, then resolves each to A/AAAA records and replaces them with IPs.
ptr	PTR lookup (deprecated)	It’s avoided altogether because it’s slow and unreliable.
exists:	A lookup	Evaluates the condition and is often avoided during flattening.
ip4/ip6	None (static IPs)	It’s already IP-based, so no flattening is needed.

This also reduces the DNS query load, leading to faster email evaluation, fewer chances of DNS failures, and fewer queries to DNS servers. Moreover, receiving mail servers find it easier to evaluate a flattened SPF record as it includes a static list of IPs and not dynamic. 

Evading the risks with AutoSPF’s flattening tool

If a service provider (e.g., Google, Mailchimp) changes their sending IPs, your flattened SPF record can become outdated, causing mail delivery issues. But with our automatic SPF flattening tool, you can manage this risk and stay within the lookup limit. Our tool works by combining IP ranges to reduce the number of entries.  Want to know more about how we can help? Contact us.