The problem
===========
If a spider started to crawl your website BEFORE you enabled prevent Spider Sessions, what can happen, is that the (previously) harvested URLS with SIDs in
them will show as results in search engines. Afterwards, often many months later, you
will still see the spider trying to access the the URLs it harvested earlier with the
session ID in it.
In summary, URL's with sessions ID's were harvested PRIOR to any session disabling, and
therefore these URL's are now indexed in search engines, and the spiders continue to
re-visit your website using the URL's with the 'osCsid' in them and you will get duplicated pages in thesearch engine.
The solution install Spider Session Remover
===============================
This uses Apache mod_rewite to look for specific spiders, and remove the session (osCsid) from the URL, and return a '301' back to the spider.
Basically, if the spider tries to do this:
GET /www.example.com/product_info.php?products_id=24&osCsid=ac8d8926059625ecb8dd9115f91d5f8a
the Apache mod_rewrite will rewrite the url to be:
GET /www.example.com/product_info.php?products_id=24
and also return a "301" (Moved Permanently) to the spider.
So over time all your duplicatd content is removed.