Reading Time: 5 minutes
One of the things I’ve learned both from self-hosting a server inside my home and from using commercial hosting is that I want bad actors as far from my server as possible. When I migrated my hosting company recently, it was primarily for cost reasons. I was delighted, though, to find that they had substantially better email and spam handling available within their service. It’s changed how I use email.
It’s not just the threat that bad actors may pose: exploiting a database on my website, for example. It’s that every interaction with my server comes with a transaction cost. When an AI scraper tries to visit every page, when someone automates an attack against the server, they use server resources. It might max out the bandwidth I’m allocated or the server RAM or processing resources on my account.
This has lead to my hosting companies sometimes sending me warning messages. Or forcing me to move to a higher bandwidth account level to accommodate the activity. It’s frustrating because you need to keep your website accessible for real people—like you!—and that means controlling resources so that legitimate visitors get through.
Like a Website But Different
I have posted before about this before where it comes to moving requests away from a website: I use Cloudflare to shut down as many bad actors as possible before they hit my server. It reduces, but does not eliminate, the likelihood that automated tools will overwhelm my resources. Cloudflare has built-in tools to block AI harvesters. I use server-side tools like .htaccess files but even that requires my server to do the work.
It’s one reason I’m frustrated with our open access repository, Elsevier’s BePress Digital Commons. While our site has statistics, we can see that the heaviest users are scrapers and so that calls into question our download and view data. Platform providers should be aggressively inhibiting scraping or provide tools to better extract the usage data so that scrapers can be excluded from metrics. It is technically easy and the resources are available.
Recently Cloudflare added what they call the AI labyrinth, which catches AI scrapers and sends them into a content honey pot. These are all available for free and help me feel like my content is a bit more secure as well as keeping those scrapers off my website. It feels safer than resorting to a zip bomb, although I’m tempted (here are instructions).
But the free plan I have with Cloudflare does not help me with spam (they have email tools but not on the free plan). I have long had an email account for my domain name (david@ofaolain.com) but rarely used it because it required me to spend a lot of time clearing out spam. My web hosts (multiple ones) used SpamAssassin but it was never quite able to catch everything. Additionally, spammers have found creative ways to get around spam filtering. The one thing that would have helped was a more aggressive but pre-configured spam filter tool.
Pre-Treating Incoming Email
The amount of spam wasn’t overwhelming on my personal account but it was enough. And, frankly, learning to master SpamAssassin was not something I really wanted to spend time on. It was a level of tinkering that might or might not pay off. There are ten levels (10 is more lenient, 1 is most strict) in the configuration tool that is outward facing. There appears to be no easy way to configure based on content of a message. This meant that, to a certain extent, you had to play whack-a-mole as each new email with the same spam came with a slightly different address.
My primary mail account has been on GMail. You can block specific addresses and I’ve used rules to filter out rubbish. I have been continually surprised at how much obvious spam gets through Google’s spam filters. I have seen a lot of GMail.com addresses and I wonder if they give their own domains a bye.
But Google’s servers are not my servers. I wanted something that got rid of everything. This would require something more than SpamAssassin. So I was delighted to find that my website hosting company had just licensed Mailchannels for its customers. Mailchannels provides inbound and outbound mail filtering. The CPanel configuration took me a bit of effort because it assumed my website host was managing my DNS (so I needed to make changes with Cloudflare) but once it was working, I was in good shape. In my defense, the documentation for this is pretty poor and it isn’t clear which port needs to be defined (or if one needs to be defined at all). In the end, I found someone else’s post while I was waiting for technical support.
I use Cloudflare to manage my domain name and so it is also my domain name server (DNS) provider. This meant that, instead of my domain mailbox pointing at my website host, I needed to divert it to point at Mailchannels. In that way, my email would flow through Mailchannels first and get pre-treated.
Before I switched web hosts, I tried an alternative email platform. Similar to what I describe here, I pointed my Cloudflare DNS to the new email provider. The one I selected was Tuta.com. But its spam filters were hardly better than what I was already experiencing. In addition, there was no way to keyword filter email. It also added an additional annual fee to run my custom domain through their system. I cancelled after about trying it for a week.
My website host also provided a service called Boxtrapper. Unlike Mailchannels, it is an app that works on email that has already arrived at my email server. I turned it on as well to see how much I could tighten the screws. The graphic below shows how an email flows through from the internet to my inbox with both Mailchannels and Boxtrapper engaged.
It worked a treat. Not unexpectedly, the first couple of days required some monitoring. I found that Mailchannels did a good job of flagging emails that are spam. As you can see in the log file below, it was blocking a lot of emails. I really like the transparency though, so that I can see what is being blocked and why. I can allow or disallow addresses in case Mailchannels gets an email wrong. For example, in the log below, a spam email was delivered. I manually blocked it and hopefully will see some machine learning happen.
This is the commonality among all of these: there are very few word-based filtering tools. I am not sure if they would help much—would I spend as much time crafting them as just blocking new email addresses—because common spam words are like common email domains. You may over-filter just as you may over-block.
The value to me of the Mailchannels service is that it is doing something technical like SpamAssassin without me having to know what it’s doing or how to configure it. The other thing that is helpful is that it provides a more legitimate mailer than I experienced with my last website host. This matters because, when I send an email, the recipient’s email server is likely to go through the same process as I am now using. The Mailchannels outbound email server is more trusted than my own domain or the older mailer I had access to with my last website host. This should mean fewer rejected newsletter emails!
I did have to update my DMARC, SPF, and DKIM information just as I had when I switched from Automattic’s JetPack plugin to the Newsletter newsletter plugin. This was easy and Mailchannels provides information as part of the configuration process. My website host also has testing tools, so I was able to verify it had worked, as you can see in the email header below.
I started with both Mailchannels and Boxtrapper. This is probably the best configuration for me but it also created more friction. Boxtrapper works by holding all email it receives and sending a request for confirmation back to the sender. Ideally all humans would initiate the confirmation and Boxtrapper would then release the email.
Some emails you receive, though, can’t get returned to a real person. This means you still need to check the Boxtrapper queue to see what is in there and release the good email so that it can reach your inbox. Boxtrapper works great—it caught the one spam email Mailchannels missed in the log above—but it is another place to look for email. In the end, given what I have seen with Mailchannels, I turned off Boxtrapper. If the spam grows beyond one email every week or so, I can always re-engage it.
I don’t get enough email, spam or otherwise, to have it take up resources in the same way that website activity does. I have not experienced it but I’m confident that a sudden burst of emails would trigger whatever machine learning is involved by both Mailchannels and Spam Assassin. But it is still nice to know that, if an email makes it to my inbox, it has run a gauntlet of tests to ensure it’s most likely an email I want to receive.
The benefit of being able to use my own email is, like in so many other cases, I can start to drop commercial providers. I would like to move off Google’s mail and other products entirely, but email has been a sticking point. Now that my own email server is in good order, I feel like I can start to bring my reliance on Google (and Microsoft) to an end.