An unofficial blog that watches Google's attempts to move your operating system online since 2005. Not affiliated with Google.

Send your tips to gostips@gmail.com.

October 8, 2007

Search Engine Comparison Poll: The Results

Six days ago, I posted a poll that asked you to evaluate the quality of the first results from Google, Yahoo, Windows Live. You had to enter a query you were particularly interested in and decide which result is the best. If you found two or three equally good results, you were allowed to select all of them.

Obviously, the poll doesn't have any scientific value, but it's still interesting to see that Google doesn't have an impressive score (in only 51% of the cases, Google delivered the best result), while Windows Live and Yahoo have similar scores: 35% for Windows Live and 30% for Yahoo. It's also interesting to see that these results were almost constant every day of the experiment, so there might be some truth there.

While I didn't monitor the correlation between queries and votes, here's a list of some of the most frequent queries: Google, test, directoryEntry, Linux, Microsoft, Radiohead, cow, Gmail, Java, search, sex, "Magnus Franklin", The Torah and the West Bank, art Liverpool, sweet krissy, quchen, server based solutions, ruby, sweet krissy.

28 comments:

  1. That was one of the two surprises. Apparently, Microsoft did a pretty good job with the latest update. The other surprise was Google's underwhelming result.

    ReplyDelete
  2. not a surprise for me, i've been using live together w/ google since the new revamped live launched a while ago. however, the poll was a great initiative. it keeps us aware that google is not everything...

    ReplyDelete
  3. Not surprising except Yahoo is lower in rankings than LIVE. Yahoo is much better in my opinion.

    ReplyDelete
  4. A 51% "success rate" could mean Google is "only" the best 51% of the time, or it could mean Google returns exactly the right result for the tastes of 51% of the people, and that the other people prefer other types of results. E.g. maybe a typical Yahoo user is phrasing things differently to return "good Yahoo results" -- each search engine may have their own syntax peculiarities which searchers adjust to in order to retrieve best results.

    ReplyDelete
  5. It's funny to see sex as a popular search term on google operating system

    ReplyDelete
  6. 51% vs 35% ??
    Sounds to me that Google is 16% better than Live. Nothing to sneeze at, I'd say.

    ReplyDelete
  7. > 51% vs 35% ??
    > Sounds to me that Google is
    > 16% better than Live. Nothing
    > to sneeze at, I'd say.

    Actually, by this statistic, it's possible Google may have been better than Live 65% of the time. Because while Yahoo stole the crown 30% of the time, that doesn't mean Google was worse than MSN for those times -- it might be Google ranked second, but above MSN in the cases where Yahoo won. Anyway, the order of the poll was fixed so I think that might have introduced a pro-Live skew.

    ReplyDelete
  8. is it just me or are those percentages messed up?

    If you add up all those votes, it equals 2356, not 2008. So should these be the percetages:

    Google: 44%
    Yahoo: 26%
    Live: 30%

    All those added together now equals 100%.

    ReplyDelete
  9. Any chance of making the queries and votes available? I'd be curious to delve into the votes in more detail.

    ReplyDelete
  10. " eddie said...

    is it just me or are those percentages messed up?

    If you add up all those votes, it equals 2356, not 2008. So should these be the percetages:

    Google: 44%
    Yahoo: 26%
    Live: 30%

    All those added together now equals 100%."

    "If you found two or three equally good results, you were allowed to select all of them."

    ReplyDelete
  11. 116%? If it's because of multiple choice votes? If so, where they occurred only a fraction of the vote should have been attributed to each choice, either 1/2 or 1/3, depending on the vote being cast on 2 or 3 choices.

    ReplyDelete
  12. Sigh. So, a biased sample of users (readers of this blog) issues a biased sample of queries (not necessarily representative of actual query traffic). The results are meaningless.

    ReplyDelete
  13. Nobody pretended these results have a scientific value, but how do you explain that those values remained constant after the first 800-900 votes?

    ReplyDelete
  14. I was expecting Yahoo to be better... interesting results

    ReplyDelete
  15. I get more traffic from goggle but I get better sales from Yahoo and MSN. More of my Y and MSN traffic convert. Just my two cents.

    ReplyDelete
  16. I don't see that Google's result is underwhelming, or indicates anybody should switch. If a search engine is the best 51 per cent of the time then that should be the search engine you always use, no? (Unless there's proof that Yahoo or Live is better for CERTAIN TYPES of query.)

    ReplyDelete
  17. Unscientific results with only 2000 votes isn't the best sample. I switched to Yahoo from google 5 months ago and now, I never ever need to go to page2 of my search results to find what I am looking at. Mainly because I have adapted to the way Yahoo search works. I'd stick with yahoo search.

    ReplyDelete
  18. I would second Matt's request. Let the data loose!

    ReplyDelete
  19. Irrespective of the search engine you still get a squillion ebay threads returned. Someone should introduce an ignor ebay button - their rating would skyrocket.

    ReplyDelete
  20. What Can i do to get steady ranking from this search engines (google)?.
    cause my results page just for a week. Please help me..

    Thanks for your Answer..
    Nando Topcoy
    http://best-results.blogspot.com/

    ReplyDelete
  21. This is not a big surprise. But it would be more challenging to follow the progresses of the many small engines silently growing in the shadow of Google and Yahoo, like for example scones.free.fr/search and www.doubletrust.net, which are experimental but show some interesting new features.
    Hopefully they will lead to something instead of being silently dropped under the majors pressure :/

    ReplyDelete
  22. What's the point of comparing the top 3 search engines, let alone any search engine in general, as long as i get what i want from the search engine, i'm happy, Google or a RSS Search Engine.

    I'm 16, a public high school student, and live in a infamous county of Los Angeles, get the picture.

    ReplyDelete
  23. I think everyone's making a big deal out of nothing, just be glad we have search engines today, although this is all coming from a kid, who has little knowledge about life, but wants to learn more.

    (3,2,3) couple, Windows late 2009 or near January 2010, 4 - 6115

    ReplyDelete
  24. How do you restore the google Search Engine functions after it was affected by a virus ?
    I want to know how the google Search Engine can be restored to normal when a spyware and/or other viruses affected the system. I have been downloading some anti-spyware programs, such as Spybot, to reverse the effects and I want to know what applications I can used to turn the search engine back to normal. I have my Yahoo Search Engine results mixed with different web addresses. Thanks.

    ReplyDelete
  25. I think BING is a great Search Engine

    ReplyDelete
  26. The recent poll has totally changed results now.
    Google is now at number first because 85% searches have been done through Google and remaining 15% by from other search engines. In my opinion other search engines cannot compete with Google as it can also crawl on the comments and every single text of the webpage

    ReplyDelete
  27. Each search engine may have their own syntax peculiarities which searchers adjust to in order to retrieve best results.

    ReplyDelete