May 6, 2009

Sites that Require Registration? Use Google

Some web sites show different content depending on the referrer. News sites like the New York Times, which display many of the articles only if you are a subscriber, make an exception if visitors click on results from Google and other popular search engines.

Google has a feature called "first click free" for sites that require subscription. "First Click Free is designed to protect your content while allowing you to include it Google's search index. To implement First Click Free, you must allow all users who find your page through Google search to see the full text of the document that the user found in Google's search results and that Google's crawler found on the web without requiring them to register or subscribe to see that content. The user's first click to your content is free and does not require logging in."

While the first click is free, when you click on a link from the page, it's likely that the site will ask you to register. This is especially annoying for sites like nytimes.com, which use article pagination to increase the number of page views.

One workaround is to copy the URL of the page you can't access directly in Google's search box and click on the first result. If Google didn't index the page, you can use SearchWiki to add the page in the list of search results: make sure you are logged in using your Google account and click on "Add a result" at the bottom of Google's search results page.


Another option would be to use the credentials collected by BugMeNot.

6 comments:

  1. Refspoof for Firefox is a way to get free access to these sites. Just set the referrer to "www.google.com".

    http://refspoof.mozdev.org/

    ReplyDelete
  2. @Zachary, use RefControl for Firefox. It also allows you to spoof your referrer.

    http://www.stardrifter.org/refcontrol/

    ReplyDelete
  3. About time too - it used to really annoy me that Google provided full listings (with high ranking) to pages that then refused to let you access the indexed content, without first surrendering your personal details and agreeing to a lifetime of third-party spam.

    Go Google for listening to our complaints!

    Of course the real test is whether sites that Do Not Comply will lose their preferential listing rights. We'll see ...

    ReplyDelete
  4. Nice, but what if someone made a spoofed http request !

    ReplyDelete
  5. I understand what your saying can be done but can't seem to do it, can someone make a screencast about this?

    ReplyDelete
  6. "First click free?" Didn't that used to be called "cloaking?"

    ReplyDelete

Note: Only a member of this blog may post a comment.