How Thunderbird’s Scam Detection Works

Since upgrading to Mozilla Thunderbird 1.5 beta 2, I’ve seen a number of messages slapped with a warning label that “Thunderbird thinks this message might be an email scam.” It appears at the top of the message, in the same style as the junk mail notice bar or the warning that remote images have been blocked, and there’s a button to mark the message as “Not a Scam.”

There’s only one problem. Since SpamAssassin and ClamAV do such a good job of catching the phishing scams before they reach my inbox, Thunderbird has yet to catch any actual phish. But there’ve been a lot of false positives. It’s hit LiveJournal reply notices, newsletters from IEEE and Golden Key, a Spam Karma notice from my own blog, and I’ve seen it on both outbid notices and updates to saved searches from eBay.

I found myself wondering just how Thunderbird’s phishing detection decides that a message is suspicious—and how to teach it that the next LJ notice isn’t a scam.

The Thunderbird support website doesn’t seem to have been updated yet. Most of the articles I’ve found only talk about TB adding the feature, not how it works. The best information I found was this Mozillazine forum thread, which included a link to the actual code that makes the decision, in phishingDetector.js. Thunderbird looks at the following:

  • Links that only use an IP address, including dotted decimal, octal, hex, dword, or some mixed encoding.
  • Links that claim to go to one site, but actually go to another. (Phishers do this to fool you into going to their site. Legit mailing lists sometimes do this with redirectors for tracking purposes.)
  • Forms embedded in the email. (This explains the LiveJournal notices.)

It also appears to trap text URLs containing HTML-escaped characters, which explains the Spam Karma reports. In this case the report includes a spammer’s link with ​ in the hostname. The message is plain text, so Thunderbird leaves the entity as-is when displaying it…but decodes it when it creates the link. Result: a link where the text and URL don’t match.

The easiest way to prevent it from freaking out over the next message? Add the sender to your address book. I’m not sure that’s a great idea, since a phisher could guess which addresses you have saved and spoof them, but it’s at least simple. I guess I’ll find out whether it works the next time I get a reply notice from LJ. Update: Adding the sender to your address book doesn’t seem to have any effect.

Update 2 (July 12, 2006): The comment thread’s gotten long enough that I can see people might miss this, so here’s how to disable it:

  1. Open Options or Preferences (this will be under the Tools menu on Windows, Thunderbird on Mac, or Edit on Linux).
  2. Click on Privacy (there should be a big padlock icon).
  3. Click on the E-mail Scams tab.
  4. Disable the “Check mail messages for email scams” option and click on Close.

That’s it.

View Kelson Vibber's LinkedIn profileView Kelson Vibber's profile on LinkedIn

31 thoughts on “How Thunderbird’s Scam Detection Works

  1. bunnyhero

    i just installed the 1.5rc2 and sooo many items are being flagged as scams. what’s worse is that with a fresh profile (i.e. no junk mail training from me), a whole TON of non-junk emails were marked as junk mail! at least the junk mail filter can be trained…

    Reply
  2. Frank

    How Thunderbird’s Scam Detection Works? That’s simple. I can sum its operation up in one word: Horribly. :-)

    Nice breakdown of what little information there is available on the topic though.

    Reply
  3. Kelson Post author

    Hard to argue with that! It’s been almost 3 months since I wrote this, and I have yet to see it fire on an actual scam. Again, I’m sure that’s partly because most of the real ones are filtered out on the server before they reach my inbox, but I’ve been unable to convince it that new mailings from LiveJournal, Ticketmaster, and Travelocity aren’t scams.

    Reply
  4. Mark

    T-bird flags most of my HTML-based newsletters. Everything from TechRepublic and Lockergnome, as well as some job sites. I wish I could just turn it off.

    Anyone know if you can replace the js file with an empty file (or put in a null function) to stop it?

    Reply
  5. Kelson Post author

    You can turn it off. In the Options/Preferences dialog, click on Privacy and open the E-mail Scams tab. There’s a check box right there.

    Reply
  6. joe

    “Links that only use an IP address, including dotted decimal, octal, hex, dword, or some mixed encoding.”

    we use dotted decimal addresses for all sorts of things, internal testing, applications, etc., and thunderbird flags it all is a scam, even messages in my SENT folder.

    file under “useless”.

    Reply
  7. kdanieli

    adding the sender to your thunderbird address book does not even prevent it from flagging emails from that sender as scams. it has flagged many, many totally safe emails as scams. this protection is totally useless. it’s a joke.

    Reply
  8. lewwy

    to people like us, its a joke. However, once thunderbird gets out onto the mainstream market, that message may be the one thing that stops an unknowledgable person from buying into a scam.

    Even with the filter being this misguided, atleast it tells us that thunderbird cares about its users. What about outlook? Couldn’t get stuffed if we got screwed over.

    Reply
  9. Jesse

    You can turn off email scam warnings. It’s under options-privacy-email scams, at least in the Windows 1.5.0.4 build.

    I haven’t turned it off yet, but I ignore it. It marks all sorts of legitimate things as scams.

    Reply
  10. Qrystal

    Ahh, glad to find this discussion. I have been hoping for quite some time that the “Not a scam” button was actually doing something, but now I’m relieved that I can just turn off the scam-checking. The “Not a scam” button doesn’t play nicely with “Allow HTML Temporary” extension, and I was getting tired of the repetition of the repetition.

    Reply
  11. BobHobbit

    I think this scam detection thing fails on all fronts… not only does it mark almost all my legitimate newsletters and mailings from sites like eBay, credit cards, etc. as scams, but it fails to catch a few obvious phishing emails. Would have been a great feature if it actually worked.

    Reply
  12. Jyatushtira

    In Seamonkey on Linux, and presumably in Mozilla, and Thunderbird you can type
    “about:config” in the location bar to enable configuring many options, some of which are not found through the preferences dialogues.

    In the list of settings shown in “about:config” is one called:
    mail.phishing.detection.enabled
    Setting this to false seems to shut off the phishing detection.
    I don’t know if it works on other operating systems.

    Reply
  13. Jyatushtira

    But wait! This works! In the user.js file in your
    .mozilla or .thunderbird or whatever directory,
    add the line:
    user_pref(“mail.phishing.detection.enabled”, false);

    Reply
  14. Simon Mikkelsen

    I have only seen it flag one mailinglist – one that I publish :-)

    But if it works so badly as described, it is useless for everybody. When you cry woolf all the time.

    When the rules are so simple and general, the phishers would probably run their scams through the filter, to make sure it passes. Then the filter makes things worse by giving a false sense of security.

    Reply
  15. Charlie Sears

    The scam detection in Thunderbird is useful to me, because it finds messages where the URL doesn’t match the text. It doesn’t hit on messages from my banks, but does hit on the scams, so I keep it turned on. I get a few false positives, but not many.

    Maybe I’m using a later version of Thunderbird than described here, and perhaps some things have been fixed.

    Reply
  16. Pingback: Back to Basics: Phish by Phone | K-Squared Ramblings

  17. Helen

    I have hunted for this on Mozilla website and could not find it in ANY of their support help/forums/FAQ.

    So I can either have it OFF or reporting stuff that isnt a scam and no way of stopping it carrying on about particular emails.

    Why is there no fix for this yet?

    Reply
  18. Gus

    It’s all about adding the sender to your address book. If you are never going to send them an email (autoresponders etc), put them in a separate address book and call it something like “whitelist”. Then you only have to click “Not a scam” once and it should remember this setting.

    Reply
  19. goldi

    Well, here it is – March of 2008 – and this “feature” continues to be useless! For whatever reason, it has always (and consistently) marked one particular newsletter that I get as a possible email scam. The first thing I tried was adding the newsletter email to my addy book. Nope, still coming through flagged!

    Finding the place to turn this “scam-checking” feature off was like looking for a needle in a haystack, until I found this blog entry. Thanks for the VERY useful info!

    Reply
  20. Daryl

    Thanks very much for the instructions on how to disable it – I only get this email scam warning on one kind of email, the daily Bible reading notes I get through my inbox every day ><

    Cheers

    Reply
  21. Roxanne

    Well…here it is March of 2009 and Thunderbird’s scam filter is still a pain in the a$$!! I am so glad I found your post to figure out how to turn it off. I just recently began using Thunderbird and googled to find this post. Thanks a bunch.

    Reply
  22. CJ

    I love how everyone is saying how ‘stupid’ and ‘useless’ this is. My opinion is, as lewwwy put it, is that anything that helps less experienced users from falling victim is better than nothing.

    In a large scale interactive environment, applications can only hold your hand so far, and nothing will replace learning the culture and ways of the internet, just like you learn how to pick scam artists in the street. What Thunderbird is doing is giving a headstart on that.

    Reply
  23. David

    All I can say is that if you follow some of these simple rules stated out in the blog post you will find that your email is less likely to be marked as a possible scam.

    For those who are having trouble, take a look at the subject line and the actual content. What links you go to.

    Ideally, it should all go to the same address.

    Although I have had the occasional email marked as a scam, I can easily overcome it by following these rules.

    Great Post!

    Reply
  24. William Furr

    Thanks for the tip! I’d prefer to be able to whitelist an individual sender, but disabling the feature entirely works as well in my particular case.

    In fact, it’s the email reports from our organization’s spam filtering software that get marked as possible scams, because the link to ‘delete all suspected spam’ is to a local IP address. Pretty ironic. :)

    Reply
  25. Kagehi

    Yeah. I find it bloody stupid too. There are **only** two things it ever marks as scams:

    1. Emails from Eve Online.
    2. Emails from Second Life, but only if it contains a URL for playing streaming media, such as a DJ announcing that they will be streaming in sim X, at Y time, and connect to Z address to hear it, if you are not in the sim.

    In terms of actual scams… Its like fracking using Hotmail. Hotmail also has this problem. I have gotten dozens of emails from viagra sellers, a few from the “I just found money in my sock, but the Zipfordian government wants it, give me an account number and you can keep some of it!”, sort of BS, etc. All of them getting through the damn filters. What does get trapped? Umm… In the case of Hotmail it always seems to, invariably, be new emails from someone that may *vaguely*, in some fashion, compete with one of their products… Gosh.. Wonder how that happens…

    Near as I can figure, Microsuck must get kick backs from Nigerian scammers and viagra people, or they just haven’t found a competing product to sell, which would require blocking everyone else selling the these things. lol

    But, yeah. Having Thunderbird pull the same stupid BS, and not give me any way to say, “Stop doing this from servers belonging to these people!”, is just irritating.

    Reply
  26. Tim Draper

    Unbelievable that Thunderbird’s only options are “always on” and “always off”. So if you constantly get emails from one account that aren’t scams and want to turn the warning off for that address only, you turn off ALL warnings. Seriously? /sigh

    Reply

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>