With the latest updates from Yahoo and Windows Live, Google's competition is getting better. Instead of asking you which one is the best, I wrote a small script that displays the top results from the most popular three search engines for a query. You should only say which result is more relevant.
Think of a complicated query from a domain you're familiar with, because you can only vote once. Select more than one option if the best result is displayed more than once. Search results will open in a new tab or in a new window. If you read this in a feed reader, go to the original page to see the poll.
Note: the order of the search engines is fixed, so the third result always comes from the same search engine. Even if the order can be easily guessed by doing some searches or looking at the code, please don't post it in the comments.
Update: The poll is now closed. Check the results.
Very interesting idea. Can't wait to see the results!
ReplyDeleteWonderful little script you have there - I'll leave my snooping comments un-noted :)
ReplyDeleteThis was no-brainer - two of them displayed more or less irrelevant results. Happy to see Google still gives the most relevant results.
ReplyDeleteDigg it.
ReplyDeletehttp://digg.com/software/Comparing_the_Top_Three_Search_Engines
I love it!
ReplyDeleteI've been using a different search strategy, I start by searching the blogosphere and find that works better than searching the whole web. Choice #2 for me was Google, but from your nifty "Pepsi Challenge", I found I like an alternative better:
http://smoothspan.wordpress.com/2007/10/03/take-the-pepsi-challenge-for-search/
Interesting. Reminds me of the Rusty Brick search engine challenge from a couple years ago: http://www.rustybrick.com/rustysearch-results.php
ReplyDeleteBarry made the results available for downloading (queries + votes, with IP addresses hashed out). Would you be willing to do that too, just so that folks can dig into the data in more detail later?
Well there is no best no 1 - if you would like to compare the the 3 engines you have to compare each ones first 3 to 5 results. My favorite would then be the 3 results you showed because they are the perfect mix. But none of them do have this 3 results.
ReplyDelete1. I want a result as a information source
2. I want a result of an important product
3. I want a result of wiith a big authority
that in the 5 first results. None of the search engines can provide me that. The first search enging that can do that will win.
Reto Hartinger - former partner at search.ch (sold the company to swiss post)
Are you sure the order of the search results is fixed?
ReplyDeleteBecause in my repeated clicking of the search button, I am get the same 3 results in a sometimes different order.
My guess is that the order changes when one or more search engine takes too long to return the results.
But sadly this means inaccurate poll results.
third one is yahoo :)
ReplyDeletehttp://www.maygle.com
ReplyDeletemake comparison more easy.
Bob Warfield may be confused
ReplyDeleteWhen doing a poll, having a fixed order of choices risks skewing the results (not that I'm complaining, I also have a fixed order of results at Blogoscoped polls I think). For better results, you need to vary the position each search engine gets.
ReplyDelete@Matt Cutts:
ReplyDeleteI tried to use Google Analytics for this, but for some reason urchinTracker gave an error in Firefox.
@abemore:
Thanks. There was a small bug, but I fixed it. It shouldn't have affected more than 1% of the people.
@Philipp:
That's what I intended to do, but I used Blogger's polls, so the order had to be fixed.
Don't know if #2 is google, but it was far and away the best result. I was looking for a legal document and it gave me the exact document I was looking for. The others both were to an article on the Act, but with no link to the actual text of the act. Of course, a non-legal person may have preferred the article, but that was not what I was looking for. Go #2
ReplyDeleteI know which one is google!
ReplyDeletewell, it's fairly obvious that to find which is which, one has only to enter the search term into each search engine, and look at the first result. to save some people trouble, the one that is currently doing the best is Google.
ReplyDeleteAlex you could have made the script less readable.
ReplyDeleteI just tried my name and none of the three results was what I was expecting.
ReplyDeleteAnd so I tried the same key word in all the three search engines separately.
Must say, the search results thrown by Yahoo were much better than what Google came up with.
Yahoo displayed my blog, flickr, linkedin...
Of course, MSN has still got a long way to go, just to catch up.
Not wishing to rain on anybody's parade, but you're using search.msn.com instead of search.live.com, and the MSN one appears to still be running Microsoft's pre-update algorithm; certainly search.live.com returns best results for the query I tried. I think you'll need to update the URL and start the experiment again. Why they've still got the old algorithm up there is a matter for another conversation, of course!
ReplyDeleteVery intersting and can't wait to see the results. I suspect that Google will come top. Not because they have the best results, but it is the results people are used to. Have a look on www.Webfetch.co.uk. You can see all 4 engines resuturning results on 1 page. There is also intersting overlap information on these engines. Did you know only 1.1% of results on page 1 overlap all 4 engines? http://www.webfetch.com/uk.wpro.sbox/search/help/aboutresults.htm
ReplyDeleteinteresting. can i got the code?
ReplyDelete@Mark:
ReplyDeleteI don't think that's true. The script uses a feed from search.msn.com, which returns the same results as search.live.com. I chose to use the feed from msn.com because live.com redirected to a login page when I wrote the script.
You can compare:
http://search.msn.com/results.aspx?q=office&format=rss
http://search.live.com/results.aspx?q=office&format=rss
@oden:
ReplyDeleteWell, you can get the JavaScript code and use it as you like it. The code uses Google AJAX Search API (for Google Search) and Google AJAX Feed API (for reading the feeds from Yahoo and MSN).
i mean the code for searching and comparing each other.
ReplyDelete@Ionut:
ReplyDeleteI think it's because I'm one of the 2.3% of internet users not in the US. Your suggested queries give me significantly different results, notably giving the NBC version of The Office on MSN rather than the BBC one on Live. (Again, the Live results are more relevant).
@Mark:
ReplyDeleteI'm not in the US either. I think that MSN redirects you to a localized version (UK?) or it takes into your account your location when ordering results. For me, both sites show the same results and have the same interface.
I don't think it's a good idea to change msn.com to live.com, but this is not a real experiment. Search engines take into account many factors when ordering results (localization, personalization), so it's hard to compare different resuts. It's also difficult to define relevancy and what's "the best result".
Anyway, the results seem to be the same in the past 2-3 days and MSN/Live Search is doing better than Yahoo.
@oden:
ReplyDeleteObviously, the code is available here (check the source).