Cloudmark filters the vast majority of spam with our existing filters. However, for entirely new attacks that have nothing in common with prior experience, our anti spam technology includes a real time feedback loop. The moment a new type of spam starts turning up in mailboxes this feedback is fed into our systems in two ways:
- Users click on the Report Spam button in their webmail or email client
- A diverse array of spamtraps
The Cloudmark Network Feedback System takes these spam reports and automatically configures filters to block the attack. These are deployed world wide in seconds, cutting off the attack as soon as it starts. Our automated system gets a very high success rate, but of course, we have a team of real people are continually monitoring the performance of the system and fine tuning it to achieve maximum performance. The 'duty' teams are constantly analyzing the feedback to see if there is anything that needs to be tweaked. As one of the duty team I find it fun in a weird sort of way, knowing that you are a key line of defense against the onslaught of botnets pushing pharma, porn and payday loans. And besides, it makes for some great blog posts too.
The engineer's job is complicated somewhat by having to wade through a lot of spam reports for things that are not spam to find the real attacks. Some of this is the fault of the email sender, and some of it is ignorance or carelessness on the part of the user. Of course, this has the potential to create spam filters for legitimate mail. Cloudmark has an array of techniques for successfully avoiding this which I'm can't discuss in public, for competitive reasons. Let's just say the secret sauce is highly successful, very frequently ;)
Since I can't discuss Cloudmark's secret sauce, let's talk about ignorance and carelessness, or as I like to call them, the Human Condition.
Factors that can lead to legitimate email being reported as spam are the frequency and volume with which the email is sent. Daily deal sites like Groupon, Living Social, Travelzoo, and Amazon Local Deals regularly turn up in the top ten
From: addresses in user spam reports. This is not because they are being spoofed by spammers but because they are marked as spam by people who actually signed up for these services but got tired of the daily message and haven't worked out that they can unsubscribe. For many users the
Report Spam button seems to be just a sort of delete key. It makes messages they can't be bothered to read go away, be they commercial, personal or political.
What does that
Report Spam button in your email client mean to you? This is junk, make this go away forever? Send this to the authorities and boil the sender in oil? Turn this a message a sandy color? Or perhaps simply,
Unsubscribe?
A decade or more ago, before the CAN SPAM law and modern spam filters, there may have been a reason for not clicking on the unsubscribe link in an email – it might take you to a malicious site or at minimum confirm your email address as active, resulting in more spam. These days that is just not true. If an email turns up in your inbox rather than your spam folder, then clicking unsubscribe will result in less email rather than more. Trust me on this, I've been doing it for years, it really works, and I've never hit a malicious link.
Of course, that unsubscribe link can be hard to find and can involve additional steps and delays after you click on it to get you off the list. Some unscrupulous senders don't want to make it easy to get off their lists. A recent Canadian proposal for regulating commercial email regulation originally included a requirement for a one click unsubscribe process, but this was removed as a result of pressure from certain email senders. I think this is exactly the wrong approach. Sending email to someone who is not interested in it is going to make them less likely to buy from you or vote for you, not more, and the better senders accept this.
While
Report Spam is easier than unsubscribing the path of least resistance will frequently be the route chosen by the masses.
Ignorance and carelessness are never going to go away, but wouldn't it be nice if our systems could get smarter to compensate? In particular, wouldn't it be nice if clicking on the
SPAM button sent an automatic unsubscribe request? Believe it or not at the bigger receivers, many already do. To address the human nature of using the "just make this go away" button many receivers offer feedback loops to senders so they can receive opt-outs directly in a machine readable standardized format. There is a great list senders may find useful on
wikipedia and
wordtothewise.
(Trust the engineers! Human nature... Solved by shell script. - Ed.) (Now we just have to get all the senders to sign up! - Andrew)
Chris Barton also contributed to this post.
