"The Fraunhofer Institute, an independent research firm, found that Yahoo! Mail users saw the least amount of spam out of the five providers tested, with nearly 40% less spam than Hotmail and 55% less spam than Gmail – meaning Gmail users in the study saw more than twice as much spam as Yahoo! Mail users."
If the results are accurate, then either Gmail's spam detection technology is no longer state of the art or Yahoo and Microsoft have dramatically improved their spam filters. Last time I checked, most of the messages from the inbox of my Hotmail account were spam.
John Mueller, from Google, says that Gmail's spam filter is quite effective: "When I first got my Gmail account (it seems like such a long time ago), I purposely signed up to all kinds of mailing lists to try out the filter. It seemed like a fun idea -- but then I noticed that I actually wanted to keep my Gmail account :). About one spammy message comes through each month, but the most important part is that I haven't lost anything that I was waiting for."
I rarely get spam messages that aren't flagged by Gmail, but the spam filter is not perfect. Sometimes Gmail flags legitimate messages as spam, so the algorithms are too aggressive.
What do you think? Is Gmail worse than Yahoo Mail or Hotmail when it comes to filtering spam?
Update: Here's Fraunhofer's press release with a link to the study. Unfortunately, it's in German. (Thanks, Lazlo.)
Update 2: Apparently, the study was sponsored by Microsoft and Yahoo's conclusions are misleading. The "prestigious" Fraunhofer Institute counted all the spam messages received by six users and Gmail users happened to receive more spam messages than Yahoo Mail and Hotmail users. The study admits that Yahoo Mail and Gmail detected correctly all the spam messages, while Hotmail couldn't detect 23% of the messages.
Yahoo concluded that "independent empirical studies done by the prestigious Fraunhofer Institute show that Yahoo! Mail is #1 in blocking malware and spam from reaching mailboxes. It's official. No one fights spam harder, smarter, or better than Yahoo! Mail." But so is Gmail, according to Fraunhofer's irrelevant study.
forever Gmail!
ReplyDeleteGMail spam filter is great. Since I created my account I haven't got any unwanted mail and all valuable messages weren't blocked by the filter.
ReplyDeleteGmail's spam filter is definetely better than Hotmail's, as you said in your post. About Yahoo, I have a friend who switch from Yahoo Mail to Gmail 2 weeks ago because she couldn't keep up with all the spam she was getting. And it works very well for her.
ReplyDeleteYahoo: Out of 200 new messages in my inbox maybe FIVE are actually for me.
ReplyDeleteGmail: since <2007, I can count the number of spam messages that have snuck into the inbox with one hand.
I'm not even going to talk about Hotmail. Needless to say, both are forwarded to Gmail, where everything is properly sorted :)
I had to create several filters to automaticaly get rid of russian mails ( they didn't seem to get flagged as spam )
ReplyDeleteNever used Yahoo and on my hotmail everything that's not from a trusted sender is marked as spam.
In the study, if gmail got more Spam its only because 10 times more mail goes through gmail than yahoo.
ReplyDeleteGmail is too agressive. I've had a legitimate work offer be marked as spam (which came from an address I hadn't sent anything to, but a domain where I had).
ReplyDeleteGmail is #1 at spam filtering.
ReplyDeleteWhoever did that "study" is probably still living in a world where HTML frames are cool.
@Anonymous:
ReplyDeleteI'm sure they know statistics 101.
How can we comment on a study that won't say what it studied?
ReplyDeleteGmail spam filter is the best in the world at present. I don't recall if I have ever had any spam messages. My primary email was yahoo which is full of crappy spam messages. So I decided to auto forward all my emails to gmail which, in fact, did a better job at getting rid of spam and keeping the important emails. This way I never have to open yahoo ever again.
ReplyDeletegmail spam filter rocks, hotmail and yahoo sucks...
ReplyDeleteHow can I get GMail SPAM filter to STOP sending anything but ENGLISH emails? I have bulk filtered many times DAILY and I still get them hundreds a day! Frustrating!!
DeleteFrom my experience gmail still has the better filter. I doubt the institute is independent.
ReplyDeleteNo doubt, gmail's filter is the best.
ReplyDeleteThe study is attached to this page: http://www.sit.fraunhofer.de/pressedownloads/pressemitteilungen/20100412spamStudie.jsp
ReplyDeleteCan anyone fluent in German have a look?
I do not use Hotmail of Yahoo, but it is hard to imagine that any spam filter can beat Gmail which is almost perfect... I do not remember last time I got a spam message and no wanted messages are misclassified. So the study essentially reports that Yahoo users get 55% less spam than zero spam Gmail users get! How did they calculate 55% of zero? :)
ReplyDeleteI don't see a single reference to the word "gmail" in this document.
ReplyDelete@George look for Google.
ReplyDeleteI cant stand yahoo mail UI. And last time I checked my hotmail was full of junk.
ReplyDeleteEven microsoft itself keeps sending me mails about new features and whatever... (why not just a button on top "new stuff")
Gmail spam filter is the best, if I remember right I only had to check the spam twice for messages that went there by mistake.
Used Google translate on the link above. Didn't really say much more than what was already posted.
ReplyDeleteAn idea though. I could be wrong, but it looks like the study was looking at email in the German language. Google's spam filter might work a whole lot better for English than German. I would find it surprising if Hotmail and Yahoo were actually better, but the language switch might be a factor.
@Anonymous Yes, that could be true.
ReplyDelete@Anonymous Yes, that could be true.
ReplyDelete@Anonymous: I live in Germany, and I don't have the impression that the effectiveness of Google's filter depends on the language.
ReplyDeleteIt is written that "The study was created with financial support from Microsoft, and lasted four weeks"
ReplyDeleteThe first anon is right. While they count spam messages in inbox (false negatives), they don't count anything else. Not false positives, not even the overall e-mail volume.
ReplyDeletePage two of the study mentions it was sponsored by microsoft.
Have just read the full report of the studie (url is in a comment above). One thing is very important:
ReplyDeleteThey did count Spam Mails in the Spam Folder as Spam (!), because users tend to check the Spamfolder regularly a waste time with it.
(Quote p.1:"Ebenfalls von Interesse waren die Spam-Nachrichten, die in der Spam-Box zugestellt wurden, da sie einen gewissen Zeitaufwand für den Nutzer bedeuten.")
For the final ranking, they use the number "E()", that is defined on page 36 as the number of spam mails in the inbox + spam mails in the spamfolder.
On page 65 they admit that neather in gmail nor in yahoo any spam got in the inbox.
My wife had a Hotmail account for a long time.
ReplyDeleteWe just created a GMail account for her that downloads all hotmail emails e filters the spam!
That's how much more spam hotmail has....
Counting correctly filtered emails in the spam filter as incorrectly filtered? Yes, it's about as methodologically respectable as most of the studies that come out of MS.
ReplyDeleteI translated the PDF file using Google Translate, but it's not very clear. From what I've read, six people created email accounts at Yahoo, Hotmail, Gmail, GMX and Web.de and then waited to see how much spam messages they get.
ReplyDeleteGMX - 116
Gmail - 18
Hotmail - 13
Web.de - 57
Yahoo Mail - 8
"The metric M9 consider what portion of received spam messages arrive in the inbox. In the subjects considered here have Google Mail and Yahoo! 0% arising from the peak, ie no Spam message has been received in Inbox. Hotmail have been in 23% of received spam messages in their inbox is sorted according- but it was purely of emails from Hotmail himself is- were sent." (machine translation)
So all the spam messages were correctly flagged by Yahoo Mail and Gmail. What a misleading study!
Just looked into the study (and I am fluent in German) - lots of false information here.
ReplyDeleteThe study is talking mostly about the TOTAL AMOUNT of spam that reached the test accounts, meaning the Inbox and Spam Folder counts added together.
Yahoo! got 8 spam mails in total during the study, Hotmail 13, Gmail 18. (pretty small numbers altogether by the way)
You are talking about spam mails reaching the inbox (and not being sorted out by the spam filter), which is a completely different story.
But the Fraunhofer guys also took care of this (let's forget for a second that the total numbers are too small to draw meaningful conclusions). Now we're getting totally different numbers:
Gmail and Yahoo! both had zero spams in their inboxes, taking the lead in this statistic.
Guys, please check your numbers before you start writing misleading articles like this one.
---
EDIT: Sorry, some guys in the comments beat me to it!
I do not trust any German studies including any product from Google. Germans are Apple lovers, and Google haters, And this has gotten worse since Android Phones start to push Apples iPhone market down...
ReplyDeleteI also have gmail, yahoo and hotmail accounts. Inbox in google gets a spam mail like once in a monce, and yahoo and hotmail are full of spam-mails.. there is no improvement on them at all.
I can only speak from my own experience but i conclude that gmail fights spam really well now, comparing with what was happening let's say 2 years ago. Maybe yahoo isn't too bad, but their products including mail, look and act like an old man.
ReplyDeleteI use all three email services, GMail and Hotmail for my important personal mail and Yahoo gets my mailing lists and other assorted hobby mail. Both GMail and Hotmail seem to do excellent jobs on keeping the spam out, I never see spam in the Inbox or spam folders, whereas my Yahoo account gets well over 200 spam mails filtered into the spam folder each week.
ReplyDeleteThey did a study with only 6 users? I would think you need a bigger audience before you draw conclusions on what mail system does the best job filtering spam.
Well, my gmail inbox tends to pass maybe 2-3 pieces of spam PER YEAR. I don't know if it's the best, but it's certainly more than good enough for me.
ReplyDeleteLess than a week ago, I also just signed up for hotmail, in anticipation of checking out the new features coming out later. It already has caught 14 spam messages, and let one through. I'm not impressed, especially since I haven't given out my hotmail address to anyone.
So Hotmail has a better spamfilter than Gmail? What a load of bull! Gmail passes maybe one spam message A YEAR into my inbox, in Hotmail you see the same amount in one HOUR!
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteYahoo: actually, i stopped checking my yahoo inbox as i couldn't recognize which mails r for me & which r spam...yahoo sucks
ReplyDeleteGmail: i'm with gmail for about 2 years...never got a spam into my inbox...gmail rocks
Hotmail: i don't like microsoft products
I won't use yahoo and hotmail even though they'll pay for me.
ReplyDeleteHow desperate of Yahoo! to press release such a pathetic study. I've used gmail from the every early days, and have never had a mail mislabelled as spam since the very, very early days. And I get no more than a couple of spams in my inbox per month. Last time I looked, Hotmail was thigh deep in spam - though I've heard rumours M$ has improved. Yahoo! used to frequently mislabel my emails, but I've not used Yahoo! Mail for a couple of years.
ReplyDeleteon hotmail all i receive in the inbox is spam. on gmail most of the spam is filtered. on yahoo some of the good messages get filtered too, and i had to train it filter some of the messages that were actually spam. so for my accounts the order would be 1. gmail 2. yahoo 3. hotmail. that's one of the reasons i stopped using hotmail actually. i only kept gmail and yahoo.
ReplyDeleteLately, You have been posting interesting posts comparing Gmail, Hotmail and Yahoo Mail.
ReplyDeleteI just would like everyone to remember that Gmail is the only email provider that allows you to play mp3 files you receive by email, online!
It works since long ago and it works very good.
This is an example showing the generosity and the efficiency of Google services.
Fuck Yahoo, Gmail is better FOREVER. I love my email Gmail for life .
ReplyDeleteI've had one good site flagged as spam until I white-listed it (I forgave Gmail - those emails do look like spam). Only one or two real spam slip thru each month, while the spam folder gets 100-150 new spam/day. Anyone who considers that a problem shouldn't be allowed online.
ReplyDeleteCan't believe it, I had to dig deep down to page 65 to find the most relevant sentence in this study: "Es ist jedoch bemerkenswert, dass bei allen Dienstanbietern keine externe Spam-Nachricht in der Inbox gelandet ist."
ReplyDeleteIn English: "It is remarkable that at all email providers no external spam message made it into the inbox."
To summarize:
- All providers' spam filters worked equally and perfectly by filtering out 100% of external spam.
And:
- The spam numbers in the above comments and post update (23% for hotmail) are actually internal emails by the providers since these were counted as spam, too.
- The 73 page report does a great job in hiding the actual numbers of false negatives by adding up spam messages in the inbox and spam box for most part of the report.
I see from the comments that the study was severely flawed, but regardless, I thought I'd make the point that Gmail's spam filtering system seems to do a much better job if you make sure to report spam that DOES get through. It appears that Google uses the corpus of your spam vs non spam email to analyse the likelihood that an incoming email is spam. Remember, what is spam to one person may not be spam to another -- and I think Google's system takes this into account.
ReplyDeleteThis is Spam From Yahoo and Microsoft !
ReplyDeleteThe Fraunhofer study has been financed by Microsoft and Yahoo and Microsoft came first.
ReplyDeleteNo further comments .....
Funny, I get no spam in my Gmail, so Hotmail and Yahoo! must be so frikkin' awesome they actually send the spam back to the spammers!
ReplyDeleteIs there a reason this article is the same word for word as the one found at http://techglow.co.cc/gmails-spam-filter-no-longer-effective ? Either you posted to both places or someone is plagiarizing.
ReplyDeleteI have no connection to that site.
ReplyDeleteGoogle's Mail is far better than Yahoo's mail and ob.. hotmail.
ReplyDeleteGMAIL rocks...
I will go with Gmail. I have been using it for more than 6 years and I am glad that I stopped using yahoo mail. Gmail is 5 times faster and as far as spam is concerned, it has done a pretty well !
ReplyDeleteI've had my gmail account since it was first released and I am a huge user.. lots of newsletters and prescriptions, forums, etc.. and I rarely get a piece of spam. In fact, I think the last time I did was months ago.
ReplyDeleteGmail is the best (definitely the coolest, too) and all the others are just posers.
Is this the Microsoft way of things now? Sponsoring lies and being deceptive?
ReplyDeleteI run a Twitter account that tweets out gems of spam mail subjects.
ReplyDeleteI take most of them from Postini, a Google-owned product that focuses on spam filtering among other things. Postini does a good job.
Oddly enough, messages that make it through Postini still get filtered as spam within Gmail. Never figured that one out.
As for the comment about legitimate messages getting filtered as spam (i.e. false positives) this can be due to content, due to other people flagging the message as spam (in the case of mass sends), due to the reputation of the sending IP address, or even other factors.
All in all I'm impressed by spam filters at all of the major ISPs. It was just a few short years ago that A LOT of spam made it through. Now it is the exception instead of the norm.
My experience of using both Yahoo Mail and GMail for the past several years is that Yahoo let's through a great deal more spam than does GMail. Rarely a piece of spam gets into my GMail inbox and I've had no legitimate emails go into the spam box. Quite the contrary with Yahoo. This "study" is just more BS from Yahoo and Microsquish.
ReplyDeleteAnd, today, Hotmail offers me up lottery winnings! You know, 'cause I may already be a winner from the Spain lottery (even if I never bought a ticket!).
ReplyDeleteMyeah ... *6* clicks to mark it as spam.
Gmail spam filter has always worked well for me. I get 2 or 3 spans a week. But I don't paste my email address all over the web.
ReplyDeleteInstead I convert my email address to an URL then paste it everywhere.
Do this at http://scr.im
Mine is http://scr.im/dwight
Let them say what they want. Gmail is the best email client ever.
ReplyDeleteLet stupid Yahoo staff to be happy just for a moment. Anyway I don't need Yahoo anymore.
Fraunhofer Institute made a very bad image of themself.
ReplyDeleteThis is the very famous German precision.. haha
Well, GMail was worse like..1 year ago or something. Spam got to my inbox and it was really annoying. Eventually, they somehow made them stay in Spam folder and not reach my Inbox anymore. But, the problem with Yahoo and Hotmail is that you get a lot of false positives, and it happened very often to me to have normal e-mails in my Spam folder..Anyway, I'm still sticking with Google..
ReplyDeletewell I have both Gmail and Yahoo mail plus a pobox.com forwarding account that redirects everything to both accounts - and this allows me to see what spam is sourced directly to Yahoo and Gmail and I can say without any equivocation that Gmail does a better job.
ReplyDeleteI almost never get in my Gmail inbox - spam but when I check the SPAM folder it's chock full of crap.
About a year or two ago. this was not the case and there was about the same amount on Gmail as I now see on Yahoo.. Gmail has gotten better.
I'm sorry but i need to say that gmail is the best.
ReplyDeleteI have those three account. But i have never seen spam mail in inbox or lost any mail in span box since gmail open beta.
On the other hand, i always receive lots of spam in inbox from hotmail and yahoo.
I think no company better than gmail, it is too powerful
Never had a spam email land in my Gmail inbox. My Yahoo inbox, however, has 99% spam and maybe 1% mail actually addressed to me.
ReplyDeleteGMail is my mail client of choice because I never get spam, and never miss anything I want.
ReplyDeleteHotmail is my "trash" account - kind of a throw-way.
And my Yahoo account is about 50% spam.
GMAIL RULES!
I still wonder how a newly created email account whose address you haven't revealed to anybody can get spam. It happened to me a number of years ago and someone in this thread mention it's still happening. Does anybody here know?
ReplyDeleteJust another Yahoo fraud...
ReplyDeleteNaa no one is believing
Sorry to say yahoo mail is rubbish no. 1
@anon: Could be spammers enumerating possible accounts, trying usernames that already exist at another provider. You might not have revealed the address publicly but still shared it with people with malware. Your own provider might do non kosher things with your address.
ReplyDeleteI have definitely noticed gmail's spam filter become worse over the past month or two. I previously got maybe 1 spam every 3 months to my inbox, and usually 1 or 2 false positives every month to my spam folder.
ReplyDeleteLately, I've been getting 5-10 spam messages in my inbox every day, while still getting the occassional false positive in the spam folder.
(My overall volume of e-mail has not changed during this time.)
Up until yesterday, I rarely got any spam in my inbox. Today, though, I've already seen at least 6 spam emails in my inbox.
ReplyDeleteI don't know if Google is just tweaking the spam filter or what, but I wish they'd fix it.
I agree with the last two postings.
ReplyDeleteUntil recently, I received hardly any spam on my gmail acount.
Yesterday (June 4th 2010) I received a whole bunch of 14 spam emails.
One of them says I've won the australian lottery :-)
I've been getting many spam emails recently as well, most of them from 'hotmail' accounts. The only hotmail account I use is through a .edu for messages from school.
ReplyDeleteWhy can't we set our gmail account to get mail only from trusted senders? Why does there need to be a 'spam' folder at all.
I noticed that Gmail does not delete all spam after 30 days either, and even if it did, why would I compromise my account by not deleting spam right away?
I've got the same story going here. Until just a couple days ago, I rarely got any spam in my inbox. But the past few days it's let through about 4 per day. Very odd.
ReplyDeleteThe same here. If asked two days ago as to whether someone should pick Gmail over Hotmail or Yahoo mail, I would have said, "Hands down, YES!" But, the last couple of days have produced the most fowl spam that I have every received in my history with Gmail. Please, Google... return to the glory days!
ReplyDeleteSame here, so much gmail spam in the last few weeks!
ReplyDeleteThe false positive rate of Yahoo and Hotmail is HORRENDOUS. I can't get any email to my clients from any account anymore (even Gmail) even though my host is not on ANY blacklists.
ReplyDeleteIf Yahoo and Hotmail are catching more spam it is because they are also blocking far more legitimate messages. I wish NOBODY used these two hosts!
Gmail filter sucks for now :((( I was very happy with it for last few years :) But from last few weeks its start flooding of spam shit from all Internet right to my gmail box! :((((((((((((((( VERY DISAPPOINTED.
ReplyDeleteGoogle, fix it please!
ps: maybe we need to make new design for e-mail system for Internet (globally. like IPv6). Old scheme is too old and have poor design.
i have gmail account as well as yahoo account ,i found that gmailhas very stong spam filter compare to yahoo.....
ReplyDeleteGmail has been great for me, and then a couple days ago it started letting in about a half dozen spam messages a day after years of none at all.
ReplyDeleteI might have thaught that the above comments are from microcrap or yahoo users ,...but yea suddenly gmail spam filter is going nuts. I have getting messages like lottery spam and most spams are from gooogle groups.
ReplyDeleteBut still cant think about switching to another webmail.
I'm having the same problem. My inbox is flooded every day with spam, most of which has porn in the title. I don't get why GMail can't see this is spam, especially as there is so much of it and I mark it as spam every single day.
ReplyDeleteGmail FILTER not working...
ReplyDeletePENIS PENIS STILL COMES
VIAGRA VIAGRA STILL COMES
VICODIN VICODIN STILL COMES
LARGER PENIS STILL COMES
NOT WORDS IN THE FILTER NOT WORKING
I DO NOT HAVE TIME TO GO THOUGH OVER 100 SPAM GMAILS
GMAIL NOT ANY GOOD
I HAVE (WASTED) HOURS ON HOURS FOOLING WITH TRASH GMAIL
(ME TO ME) GMAILS...GMAIL KNOW I DO NOT SEND MYSELF ANY EMAILS...
SO HOW TO STOP (SAME) EMAILS OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER AND OVER...
I NEED TO SAY THAT OVER AND OVER
GMAIL SPAM FILTER NOT WORKING...
GMAIL NEEDS TO MAKE SOME CHANGES...
STOP THE (SAME SPAMERS)
You know why there are so many holes in the Gmail's spam filter? Because they make money selling to spammers. Why do you think there's no option to turn off the spam folder so you can't look at it? They are lying to you. After so many empty promises about their great spam filtering and "flexibility" options, why else do you think they keep letting the spam through? Otherwise they would have given us the option to just turn off the spam folder.
ReplyDeleteJust switched over from Yahoo Mail Plus (paid) to Gmail and there's no comparison - WAY less false positives and false negatives with Gmail. Actually, in 2 weeks, there have been no good emails going into the spam folder and no spam in my inbox. That's vs. an average of 4 or 5 mis-flags with Yahoo a week.
ReplyDeletegmail to stop spam will never work
ReplyDeletegoogle do not give a ship about you and spam
more spam the more google likes it
how many years now...and still penis spam
Gmail's spam filter has been losing quality for quite some time. I don't blame them, Google is in the advertising business, and a spam-blocker in Gmail is a conflict of interest. I still use Gmail, but I just manually set filters that aren't working automatically anymore.
ReplyDeleteconsidering a vast amount of scammers/spammers use yahoo accounts for spamming i personally find the findings that gmail are not good enough at blocking spam a joke! come on yahoo step up the Anti and get your hackers/spammers/scammers off yahoo and let the world have some respite from reading blah.blah from yahoo wants my bank and address details.
ReplyDelete@Anonymous November 27, 2010 1:45 PM
ReplyDelete"gmail to stop spam will never work
google do not give a ship about you and spam
more spam the more google likes it
how many years now...and still penis spam"
What kind of retarded comment is this???
(Ironically it looks like a spam mail message :P)
Seems lately the GMail spam is not working so well. My spam bucket used to only get 2 to 3 a day and now i get more than 50 spam emails a day. Used to be they never got through to me, seems gmal is slipping
ReplyDeleteI have also noticed that Gmail's spam blocking has become far less effective. Even when I specifically report a sender as spam, I still get message from that same sender (exact same email address in many cases, although some spammers use different variations of address or domain to fake out spam filters). Up until a few months ago, Gmail worked nearly flawlessly. Now, it feels like it's about 50% as effective.
ReplyDeleteYes, I have the same problem and the bulk filter doesn't seem to be working at all since the "new Look" came on board.
DeleteGmail does not have spam filters. Yahoo does and I never get spam there. I just wish Goggle would improve the spam control for gmail. Get with the times.
ReplyDelete