I am offering free pdf manuscripts for my customers via the Squirrelcart Download module.
Some of these documents are quite large.
I am seeing LOTS of downloads that appear to be crawlers like Googlebot that are downloading the docs "Not Logged In."
This is using a lot of bandwidth. I have searched for a setting or Rule I can use to block downloads "Not Logged In" but haven't found one that I recognize.
Is there something I can do besides blocking the crawlers at the OS level? I obviously don't want to block Google completely (or any other indexing service for that matter.)


Reply With Quote


