How Thunderbird’s Scam Detection Works
October 28th, 2005 by Kelson. Posted in Mozilla, Spam, Troubleshooting and tagged for detector, email, Mozilla, phishing, scam, thunderbirdSince upgrading to Mozilla Thunderbird 1.5 beta 2, I’ve seen a number of messages slapped with a warning label that “Thunderbird thinks this message might be an email scam.” It appears at the top of the message, in the same style as the junk mail notice bar or the warning that remote images have been blocked, and there’s a button to mark the message as “Not a Scam.”
There’s only one problem. Since SpamAssassin and ClamAV do such a good job of catching the phishing scams before they reach my inbox, Thunderbird has yet to catch any actual phish. But there’ve been a lot of false positives. It’s hit LiveJournal reply notices, newsletters from IEEE and Golden Key, a Spam Karma notice from my own blog, and I’ve seen it on both outbid notices and updates to saved searches from eBay.
I found myself wondering just how Thunderbird’s phishing detection decides that a message is suspicious—and how to teach it that the next LJ notice isn’t a scam.
The Thunderbird support website doesn’t seem to have been updated yet. Most of the articles I’ve found only talk about TB adding the feature, not how it works. The best information I found was this Mozillazine forum thread, which included a link to the actual code that makes the decision, in phishingDetector.js. Thunderbird looks at the following:
- Links that only use an IP address, including dotted decimal, octal, hex, dword, or some mixed encoding.
- Links that claim to go to one site, but actually go to another. (Phishers do this to fool you into going to their site. Legit mailing lists sometimes do this with redirectors for tracking purposes.)
- Forms embedded in the email. (This explains the LiveJournal notices.)
It also appears to trap text URLs containing HTML-escaped characters, which explains the Spam Karma reports. In this case the report includes a spammer’s link with ​ in the hostname. The message is plain text, so Thunderbird leaves the entity as-is when displaying it…but decodes it when it creates the link. Result: a link where the text and URL don’t match.
The easiest way to prevent it from freaking out over the next message? Add the sender to your address book. I’m not sure that’s a great idea, since a phisher could guess which addresses you have saved and spoof them, but it’s at least simple. I guess I’ll find out whether it works the next time I get a reply notice from LJ. Update: Adding the sender to your address book doesn’t seem to have any effect.
Update 2 (July 12, 2006): The comment thread’s gotten long enough that I can see people might miss this, so here’s how to disable it:
- Open Options or Preferences (this will be under the Tools menu on Windows, Thunderbird on Mac, or Edit on Linux).
- Click on Privacy (there should be a big padlock icon).
- Click on the E-mail Scams tab.
- Disable the “Check mail messages for email scams” option and click on Close.
That’s it.
Related Posts
- Outlook Viruses Trash Non-Outlook Mailboxes
- Back to Basics: Phish by Phone
- Null Notice
- Flagging (Non)-Spoofed Mail
- Another bogus warning
25 Responses to “How Thunderbird’s Scam Detection Works”
By Bob on Nov 30, 2005
That ClamAV link should be .net. The .org address you have is, uh, something else.
By Kelson on Nov 30, 2005
Oops! Thanks for catching that! Fixed!
By bunnyhero on Jan 4, 2006
i just installed the 1.5rc2 and sooo many items are being flagged as scams. what’s worse is that with a fresh profile (i.e. no junk mail training from me), a whole TON of non-junk emails were marked as junk mail! at least the junk mail filter can be trained…
By Frank on Jan 18, 2006
How Thunderbird’s Scam Detection Works? That’s simple. I can sum its operation up in one word: Horribly.
Nice breakdown of what little information there is available on the topic though.
By Kelson on Jan 18, 2006
Hard to argue with that! It’s been almost 3 months since I wrote this, and I have yet to see it fire on an actual scam. Again, I’m sure that’s partly because most of the real ones are filtered out on the server before they reach my inbox, but I’ve been unable to convince it that new mailings from LiveJournal, Ticketmaster, and Travelocity aren’t scams.
By Katie on Jan 18, 2006
With the service fees Ticketmaster charges, I’m not convinced it’s not a scam.
By Kelson on Jan 18, 2006
Good one!
By Mark on Jan 26, 2006
T-bird flags most of my HTML-based newsletters. Everything from TechRepublic and Lockergnome, as well as some job sites. I wish I could just turn it off.
Anyone know if you can replace the js file with an empty file (or put in a null function) to stop it?
By Kelson on Jan 26, 2006
You can turn it off. In the Options/Preferences dialog, click on Privacy and open the E-mail Scams tab. There’s a check box right there.
By joe on Jan 30, 2006
“Links that only use an IP address, including dotted decimal, octal, hex, dword, or some mixed encoding.”
we use dotted decimal addresses for all sorts of things, internal testing, applications, etc., and thunderbird flags it all is a scam, even messages in my SENT folder.
file under “useless”.
By kdanieli on Feb 8, 2006
adding the sender to your thunderbird address book does not even prevent it from flagging emails from that sender as scams. it has flagged many, many totally safe emails as scams. this protection is totally useless. it’s a joke.
By lewwy on May 26, 2006
to people like us, its a joke. However, once thunderbird gets out onto the mainstream market, that message may be the one thing that stops an unknowledgable person from buying into a scam.
Even with the filter being this misguided, atleast it tells us that thunderbird cares about its users. What about outlook? Couldn’t get stuffed if we got screwed over.
By Jesse on Jun 8, 2006
You can turn off email scam warnings. It’s under options-privacy-email scams, at least in the Windows 1.5.0.4 build.
I haven’t turned it off yet, but I ignore it. It marks all sorts of legitimate things as scams.
By Qrystal on Jun 27, 2006
Ahh, glad to find this discussion. I have been hoping for quite some time that the “Not a scam” button was actually doing something, but now I’m relieved that I can just turn off the scam-checking. The “Not a scam” button doesn’t play nicely with “Allow HTML Temporary” extension, and I was getting tired of the repetition of the repetition.
By BobHobbit on Jul 2, 2006
I think this scam detection thing fails on all fronts… not only does it mark almost all my legitimate newsletters and mailings from sites like eBay, credit cards, etc. as scams, but it fails to catch a few obvious phishing emails. Would have been a great feature if it actually worked.
By Jyatushtira on Jul 12, 2006
In Seamonkey on Linux, and presumably in Mozilla, and Thunderbird you can type
“about:config” in the location bar to enable configuring many options, some of which are not found through the preferences dialogues.
In the list of settings shown in “about:config” is one called:
mail.phishing.detection.enabled
Setting this to false seems to shut off the phishing detection.
I don’t know if it works on other operating systems.
By Jyatushtira on Jul 13, 2006
Sorry, I guess it doesn’t work…
By Jyatushtira on Jul 13, 2006
But wait! This works! In the user.js file in your
.mozilla or .thunderbird or whatever directory,
add the line:
user_pref(”mail.phishing.detection.enabled”, false);
By Simon Mikkelsen on Jul 22, 2006
I have only seen it flag one mailinglist - one that I publish
But if it works so badly as described, it is useless for everybody. When you cry woolf all the time.
When the rules are so simple and general, the phishers would probably run their scams through the filter, to make sure it passes. Then the filter makes things worse by giving a false sense of security.
By Charlie Sears on Aug 22, 2006
The scam detection in Thunderbird is useful to me, because it finds messages where the URL doesn’t match the text. It doesn’t hit on messages from my banks, but does hit on the scams, so I keep it turned on. I get a few false positives, but not many.
Maybe I’m using a later version of Thunderbird than described here, and perhaps some things have been fixed.
By Helen on Feb 20, 2007
I have hunted for this on Mozilla website and could not find it in ANY of their support help/forums/FAQ.
So I can either have it OFF or reporting stuff that isnt a scam and no way of stopping it carrying on about particular emails.
Why is there no fix for this yet?
By Gus on Jun 16, 2007
It’s all about adding the sender to your address book. If you are never going to send them an email (autoresponders etc), put them in a separate address book and call it something like “whitelist”. Then you only have to click “Not a scam” once and it should remember this setting.
By goldi on Mar 20, 2008
Well, here it is - March of 2008 - and this “feature” continues to be useless! For whatever reason, it has always (and consistently) marked one particular newsletter that I get as a possible email scam. The first thing I tried was adding the newsletter email to my addy book. Nope, still coming through flagged!
Finding the place to turn this “scam-checking” feature off was like looking for a needle in a haystack, until I found this blog entry. Thanks for the VERY useful info!
By Daryl on Sep 22, 2008
Thanks very much for the instructions on how to disable it - I only get this email scam warning on one kind of email, the daily Bible reading notes I get through my inbox every day ><
Cheers