Wednesday, May 5, 2010

Cuil - Smart or Stupid?

For the last two consecutive days, we had one of our subscription sites "attacked" by the Cuil spider. The spider would come to the site during peak business hours and do requests at the rate of 1-2 requests per second on average. When we analyzed the traffic it seemed to be having some sort of conversation or dialog with the search engine on our website - feeding all kinds of queries to our website search engine. I am not sure where it was picking up the search terms from. Perhaps it was using the search terms from the search results itself to be able to reconstruct a version of the content itself. Technically you can do that and it sounds kind of cool, but it seems cheap for a VC funded west coast search engine company.

0 comments:

Post a Comment