Until Google decides it's a good idea to provide search feeds or a new API, some "evil" people came up with Evil API, a replacement for Google SOAP API, and Evil RSS, a way to get feeds for search results. Both solutions are open source and use screen scraping (what else?) to get the results from Google. That means the solution is not quite OK if you look at Google TOS:
"You may not send automated queries of any sort to Google's system without express permission in advance from Google."
If you don't send too many queries and/or host the code on your server, I'm sure Google doesn't mind. Many Google Gadgets use screen scraping to get search results (and information from sites that don't provide feeds), and Google makes it easy to do that.
Homework:
1. Why do you think Google discontinued support for their SOAP API?
2. How would you convince Google to give us search feeds?
If the original call to evil API is to evilapi directly, then it'll be useful to use this.
ReplyDeleteIf we got to use this API on out site, we got to keep updating the evilAPI each time Google changes the html format (assuming evilAPI will make the quick RegExp change)
1. Why do you think Google discontinued support for their SOAP API?
ReplyDeleteSomething bigger & better is probably coming. Plus, the SOAP API wasn't very easy to work with. What the "evil" group did with EvilRSS would be a much better way to get search results quickly and in a format that more people could work with.
2. How would you convince Google to give us search feeds?
I'm guessing a new API, with integrated AdWords. This way, Google can make even more money, while providing a service programmers are asking for.
It would not be like AdSense - users would not get a portion of the AdWords revenue, because in return for showing the ads, they get to use the organic search results.
However, to get here, there's a change that has to be made to AdWords, so advertisers can choose to opt-out of this service (much like they can opt-out of AdSense).
Eric,
ReplyDeleteThat "BIG thing" is the AJAX search API.
They want to clear the way for The Google AJAX Search API, and hence are discontinuing features that pull in search results without the page.
ReplyDeleteThey plan to introduce ads in the results from the search API as well, so it's only logical that they will remove mediums through which people could filter out those ads.
Just go over to icerocket and get your feed there.
ReplyDeleteJust use a software like iMacros to get (scrape) the results. Works very well for us :)
ReplyDeleteBen
1. Why do you think Google discontinued support for their SOAP API?
ReplyDeleteBecause they couldn't push ads down through the results
2. How would you convince Google to give us search feeds?
Why would anyone bother trying? Google's approach is to throw mud at every wall they can and hope something sticks. The SOAP api didn't stick, even though countless applications and websites were built on it, commercial demand was there for it, and hundreds of books refer to it.
1. Why do you think Google discontinued support for their SOAP API?
ReplyDeleteBecause they couldn't push ads down through the results
2. How would you convince Google to give us search feeds?
Why would anyone bother trying? Google's approach is to throw mud at every wall they can and hope something sticks. The SOAP api didn't stick, even though countless applications and websites were built on it, commercial demand was there for it, and hundreds of books refer to it.