Spam Filters Explained

What do they do? How do they work? Which one is right for me? By Alan Hearnshaw

Spam is a very real problem that many people have to deal with on a daily basis. For those that have decided to do something about it and start to investigate the options available in spam filtering, this article provides a brief introduction to your options and the types of spam filters available.

Despite the bewildering array of spam filters available today, all claiming to the best one "of its kind" there are really just five filtering methodologies in general use today and all products rely on one, or a combination of these:

Content-Based Filters

"In the beginning, there were content-based filters."

These filters scan the contents of the and look for tell-tale signs that the message is spam. In the early days of spamming it was quite simple to look out for "Kill Words" such as "Lose Weight" and mark a message as spam if it was found.

Very soon though, spammers got wise to this and started resorting to all kinds of tricks to get their message past the filters. The days of "obfuscation" had begun. We started getting messages containing the phrase "L0se Welght" (Notice the zero for "o" and "l" for "i") and even more bizarre ? and sometimes quite ingenious ? variations.

This rendered basic content-based filters somewhat ineffective, although there are one or two on the market now that are clever enough to "see through" theses attempts and still provide good results.

Bayesian Based Filters

"The Reverend Bayes comes to the rescue"

Born in London 1702, the son of a minister, Thomas Bayes developed a formula which allowed him to determine the probability of an event occurring based on the probabilities of two or more independent evidentiary events.

Bayesian filters "learn" from studying known good and bad messages. Each message is split into single "word bytes", or tokens and these tokens are placed into a database along with how often they are found in each kind of message.

When a new message arrives to be tested by the filter, the new message is also split into tokens and each token is looked up in the database. Extrapolating results from the database and applying a form of the good reverend's formula, know as the a "Naive Bayesian" formula, the message is given a "spamicity" rating and can be dealt with accordingly.

Bayesian filters typically are capable of achieving very good accuracy rates (>97% is not uncommon), and require very little on-going maintenance.

Whitelist/Blacklist Filters

"Who goes there, friend or foe?"

This very basic form of filtering is seldom used on its own nowadays, but can be useful as part of a larger filtering strategy.

A "whitelist" is nothing more than a list of e-mail addresses from which you wish to accept communications. A whitelist filter would only accept messages from these people and all others would be rejected

A "blacklist", conversely, is a list of e-mail addresses - and sometimes IP Addresses (computer identification addresses) - from which communications will not be accepted.

While this may seem like a good idea from the outset, a whitelist methodology is too restrictive for most people and, as virtually all spam e-mails carry a forged "from" address, there is little point in collecting this address to ban it in future as it is very unlikely to be the same next time.

There are bodies on the internet that maintain a list of known "bad" sources of e-mail. Many filters today have the ability to query these servers to see if the message they are looking at comes from a source identified by this Internet-based blacklist, or RBL. While being quite effective, they do tend to suffer from "false positives" where good messages are incorrectly identified as spam. This happens often with newsletters.

Challenge/Response Filters

"Open sesame!"

Challenge/Response filters are characterised by their ability to automatically send a response to a previously unknown sender asking them to take some further action before their message will be delivered. This is often referred to as a "Turing Test" - named after a test devised by British mathematician Alan Turing to determine if machines could "think".

Recent years have seen the appearance of some internet services which automatically perform this Challenge/Response function for the user and require the sender of an e-mail to visit their web site to facilitate the receipt of their message.

Critics of this system claim it to be too drastic a measure and that it sends a message that "my time is more important than yours" to the people trying to communicate with you.

For some low traffic e-mail users though, this system alone may be a perfectly acceptable method of completely eliminating spam from their inbox - one step above the "Whitelist" system outlined above.

Community Filters

"A united front"

These types of filters work on the principal of "communal knowledge" of spam. When a user receives a spam message, they simply mark it as such in their filter. This information is sent to a central server where a "fingerprint" of the message is stored.

After enough people have "voted" this message to be spam, then it is stopped from reaching all the other people in the community.

This type of filtering can prove to be quite effective, although it stands to reason that it can never be 100% effective as a few people have to receive the spam for it to be "flagged" in the first place. Just like its similar cousin the Internet black list (RBL), this system also can suffer from "false positives", or messages incorrectly identified as spam.

Hopefully you are now armed with a little more information to be able to make an informed decision on the best spam filter for you.

For further information, consider reading the reviews and articles found at http://www.whichspamfilter.com

Alan Hearnshaw is a computer programmer and the owner of http://www.whichspamfilter.com, a web site which conducts weekly in-depth reviews of current spam filters, provides help and guidance in the fight against spam and provides a useful community forum. [email protected]

In The News:

This RSS feed URL is deprecated, please update. New URLs can be found in the footers at https://news.google.com/news

Fossbytes

8 Best Android Call Blocker App List For 2018 | Filter Spam Calls
Fossbytes
Call Blacklist is a powerful Android call blocker app that works both as a spam call blocker and SMS blocker. You can block calls and messages from any specific, private or hidden numbers, or any other number that you have not stored in your contacts ...


ZDNet

Google cuts fake ad blockers from Chrome Store: Were you among 20 million fooled?
ZDNet
The authors of fake extensions are also using keyword spam in the extension description to get a top ranking in the Chrome Web Store for searches for 'adblocker'. "Instead of using tricky names, they now spam keywords in the extension description to ...
20 million Chrome users are fooled into downloading fake ad blockersDigital Trends
Google has kicked five malicious ad blockers off the Chrome StoreTechRadar
Google Boots Fake Ad Blockers From Chrome Web StoreGizmodo
TechSpot -TrustedReviews -IT News Africa
all 31 news articles »

The Star Online

Adguard claims 20 million Chrome users installed (realistic-looking ...
The Star Online
Adguard identified five different fake ad blockers that had targeted Chrome users; Google has since taken down them down.

and more »

Cult of Mac

Hiya Caller ID and Blocker keeps robocallers at bay [50 Essential iOS Apps #3]
Cult of Mac
Knowing when to answer a call from an unknown number is a game of chance. It might be a family emergency, or it might be a scammer trying to rip you off. Hiya Caller ID and Block is a simple iOS app that attempts to determine whether incoming phone ...


WLTX.com

Cyber Crooks Take Robocalls to the Next Level
WLTX.com
The FTC is trying to figure out how to block these calls, but with cheap technology allowing scammers to place millions of calls from other countries, it's hard to do. "It's annoying." So what can you do? Number one: Download a free or paid ...


TechTarget

What is spam filter? - Definition from WhatIs.com
TechTarget
A spam filter is a program that is used to detect unsolicited and unwanted email and prevent those messages from getting to a user's inbox. Like other types of filtering programs, a spam filter looks for certain criteria on which it bases judgments ...

and more »

KCRA Sacramento

How to stop annoying spam calls
KCRA Sacramento
... ways to block unwanted calls. 2. Check if your phone has built-in protection from spam calls - iPhones can block specific phone numbers. 3. Download apps like TrueCaller or Hiya, which block robocalls or send them to voicemail. 4. Check out cloud ...


ZDNet

Windows warning: Tech-support scammers are ramping up attacks, says Microsoft
ZDNet
Microsoft says it received 153,000 reports in 2017 from customers who'd come in contact with tech-support scammers via a cold call, spam, or the web. The reports from customers last year were up 24 percent on 2016, with filings coming from 183 ...

and more »

WJCL News

2-year-old boy stands in for Marine brother who couldn't attend prom with girlfriend
WJCL News
1. Ask if your mobile phone service provider offers ways to block unwanted calls. 2. Check if your phone has built-in protection from spam calls - iPhones can block specific phone numbers. 3. Download apps like TrueCaller or Hiya, which block robocalls ...

and more »

HackRead

Over 20 million Chrome users have installed fake malicious Ad Blockers
HackRead
Ad Blockers play a vital role in hurting the earnings of a website at the same time it lets users browse the Internet without the hassle of closing irritating popup ads and getting redirected to scammy sites that bombard users with spam during the ...

Google News

Ignoring These Tips Could Result in an Inbox Full of Spam

Although there still seem to be some differences among the... Read More

Do You Know Whats in Your Trash?

A hearty welcome to all the spam fighting filters and... Read More

Demand for Spam? It exists

Do you like spam? No, I'm not kidding. Everybody knows... Read More

Spam Filters Explained

What do they do? How do they work? Which one... Read More

Quickly Eliminate 100% of Your Junk Email

Why do so many people think I need to take... Read More

Phishing for an Identity

Phishing is rapidly becoming on the largest threats to your... Read More

The Anti Spam Challenge ? Minimizing False Positives

Email is the quintessential business communication tool, so when it... Read More

FTC Botches Fight on SPAM, Microsoft Takes Over the Battle

While the Federal Trade Commission is busy fighting over definitions... Read More

How To Analyze A Rip-Off Scheme

This review is taken DIRECTLY from a piece of "junk... Read More

How To Stop Spam

I imagine you have seen, heard about, or already know... Read More

Challenge Response Spam Filters Explained

As the flood of spam increases end users are looking... Read More

Dealing With SPAM - An E-mail Address Strategy

With SPAM being such a problem it might seem the... Read More

The Vanishing Mail

Am I Just Being Paranoid Or Are The Robots Out... Read More

Im Guilty Until Proven Innocent

No doubt about it."Spam" (unsolicited commercial email) threatens to paralyze... Read More

Avoid, Shun, Thwart, Prevent, and then Filter Spam

Email is rapidly becoming the standard means of communication among... Read More

Are You A Spam Zombie?

Over the past few years you've all become familiar with... Read More

How Spammers Fool Whitelists - And How to Stop Them

Effectively stopping spam over the long-term requires much more than... Read More

5 Zero-Cost Spam Prevention Tools For All Situations!

Anyone who uses email knows what Spam is!It's annoying and... Read More

Sick Of Wading Through Spam?

It's a nightmare isn't it? You fire up your email... Read More

Blackhole or Fail - Which One Is Better For Your Mail Server?

Very often SPAMMERS take advantage of catch-all email setup on... Read More

What to Do to Avoid Getting Banned

How would you feel if you found a link on... Read More

Invasion of the Email Snatchers

They're sneaky. And stealthy. They're quiet and mostly unobtrusive, but... Read More

Email Chain Letters - Harmless Fun or Not?

I'm sure I'm not the only person on the planet... Read More

Stop Spam: How To Escape The Spam Hell-Hole

If you're anything like me, you're pretty sick of it,... Read More

20 Words That Kill - At Least When It Comes to Spam Filters

Spam, spam, spam. It's terrible not only for those of... Read More