An unofficial blog that watches Google's attempts to move your operating system online since 2005. Not affiliated with Google.

Send your tips to gostips@gmail.com.

October 2, 2007

Comparing the Top Three Search Engines

With the latest updates from Yahoo and Windows Live, Google's competition is getting better. Instead of asking you which one is the best, I wrote a small script that displays the top results from the most popular three search engines for a query. You should only say which result is more relevant.

Think of a complicated query from a domain you're familiar with, because you can only vote once. Select more than one option if the best result is displayed more than once. Search results will open in a new tab or in a new window. If you read this in a feed reader, go to the original page to see the poll.

Note: the order of the search engines is fixed, so the third result always comes from the same search engine. Even if the order can be easily guessed by doing some searches or looking at the code, please don't post it in the comments.

Update: The poll is now closed. Check the results.

27 comments:

  1. Very interesting idea. Can't wait to see the results!

    ReplyDelete
  2. Wonderful little script you have there - I'll leave my snooping comments un-noted :)

    ReplyDelete
  3. This was no-brainer - two of them displayed more or less irrelevant results. Happy to see Google still gives the most relevant results.

    ReplyDelete
  4. Digg it.

    http://digg.com/software/Comparing_the_Top_Three_Search_Engines

    ReplyDelete
  5. I love it!

    I've been using a different search strategy, I start by searching the blogosphere and find that works better than searching the whole web. Choice #2 for me was Google, but from your nifty "Pepsi Challenge", I found I like an alternative better:

    http://smoothspan.wordpress.com/2007/10/03/take-the-pepsi-challenge-for-search/

    ReplyDelete
  6. Interesting. Reminds me of the Rusty Brick search engine challenge from a couple years ago: http://www.rustybrick.com/rustysearch-results.php

    Barry made the results available for downloading (queries + votes, with IP addresses hashed out). Would you be willing to do that too, just so that folks can dig into the data in more detail later?

    ReplyDelete
  7. Well there is no best no 1 - if you would like to compare the the 3 engines you have to compare each ones first 3 to 5 results. My favorite would then be the 3 results you showed because they are the perfect mix. But none of them do have this 3 results.

    1. I want a result as a information source
    2. I want a result of an important product
    3. I want a result of wiith a big authority

    that in the 5 first results. None of the search engines can provide me that. The first search enging that can do that will win.

    Reto Hartinger - former partner at search.ch (sold the company to swiss post)

    ReplyDelete
  8. Are you sure the order of the search results is fixed?

    Because in my repeated clicking of the search button, I am get the same 3 results in a sometimes different order.

    My guess is that the order changes when one or more search engine takes too long to return the results.

    But sadly this means inaccurate poll results.

    ReplyDelete
  9. http://www.maygle.com
    make comparison more easy.

    ReplyDelete
  10. Bob Warfield may be confused

    ReplyDelete
  11. When doing a poll, having a fixed order of choices risks skewing the results (not that I'm complaining, I also have a fixed order of results at Blogoscoped polls I think). For better results, you need to vary the position each search engine gets.

    ReplyDelete
  12. @Matt Cutts:
    I tried to use Google Analytics for this, but for some reason urchinTracker gave an error in Firefox.

    @abemore:
    Thanks. There was a small bug, but I fixed it. It shouldn't have affected more than 1% of the people.

    @Philipp:
    That's what I intended to do, but I used Blogger's polls, so the order had to be fixed.

    ReplyDelete
  13. Don't know if #2 is google, but it was far and away the best result. I was looking for a legal document and it gave me the exact document I was looking for. The others both were to an article on the Act, but with no link to the actual text of the act. Of course, a non-legal person may have preferred the article, but that was not what I was looking for. Go #2

    ReplyDelete
  14. well, it's fairly obvious that to find which is which, one has only to enter the search term into each search engine, and look at the first result. to save some people trouble, the one that is currently doing the best is Google.

    ReplyDelete
  15. Alex you could have made the script less readable.

    ReplyDelete
  16. I just tried my name and none of the three results was what I was expecting.
    And so I tried the same key word in all the three search engines separately.
    Must say, the search results thrown by Yahoo were much better than what Google came up with.
    Yahoo displayed my blog, flickr, linkedin...
    Of course, MSN has still got a long way to go, just to catch up.

    ReplyDelete
  17. Not wishing to rain on anybody's parade, but you're using search.msn.com instead of search.live.com, and the MSN one appears to still be running Microsoft's pre-update algorithm; certainly search.live.com returns best results for the query I tried. I think you'll need to update the URL and start the experiment again. Why they've still got the old algorithm up there is a matter for another conversation, of course!

    ReplyDelete
  18. Very intersting and can't wait to see the results. I suspect that Google will come top. Not because they have the best results, but it is the results people are used to. Have a look on www.Webfetch.co.uk. You can see all 4 engines resuturning results on 1 page. There is also intersting overlap information on these engines. Did you know only 1.1% of results on page 1 overlap all 4 engines? http://www.webfetch.com/uk.wpro.sbox/search/help/aboutresults.htm

    ReplyDelete
  19. interesting. can i got the code?

    ReplyDelete
  20. @Mark:
    I don't think that's true. The script uses a feed from search.msn.com, which returns the same results as search.live.com. I chose to use the feed from msn.com because live.com redirected to a login page when I wrote the script.

    You can compare:
    http://search.msn.com/results.aspx?q=office&format=rss
    http://search.live.com/results.aspx?q=office&format=rss

    ReplyDelete
  21. @oden:
    Well, you can get the JavaScript code and use it as you like it. The code uses Google AJAX Search API (for Google Search) and Google AJAX Feed API (for reading the feeds from Yahoo and MSN).

    ReplyDelete
  22. i mean the code for searching and comparing each other.

    ReplyDelete
  23. @Ionut:
    I think it's because I'm one of the 2.3% of internet users not in the US. Your suggested queries give me significantly different results, notably giving the NBC version of The Office on MSN rather than the BBC one on Live. (Again, the Live results are more relevant).

    ReplyDelete
  24. @Mark:

    I'm not in the US either. I think that MSN redirects you to a localized version (UK?) or it takes into your account your location when ordering results. For me, both sites show the same results and have the same interface.

    I don't think it's a good idea to change msn.com to live.com, but this is not a real experiment. Search engines take into account many factors when ordering results (localization, personalization), so it's hard to compare different resuts. It's also difficult to define relevancy and what's "the best result".

    Anyway, the results seem to be the same in the past 2-3 days and MSN/Live Search is doing better than Yahoo.

    ReplyDelete
  25. @oden:

    Obviously, the code is available here (check the source).

    ReplyDelete

Note: Only a member of this blog may post a comment.