Remove Duplicate Content from Google/Search Engines

Remove Duplicate Content from Google/Search Engines
The problem =========== If a spider started to crawl your website BEFORE you enabled prevent Spider Sessions, what can happen, is that the (previously) harvested URLS with SIDs in them will show as results in search engines. Afterwards, often many months later, you will still see the spider trying to access the the URLs it harvested earlier with the session ID in it. In summary, URL's with sessions ID's were harvested PRIOR to any session disabling, and therefore these URL's are now indexed in search engines, and the spiders continue to re-visit your website using the URL's with the 'osCsid' in them and you will get duplicated pages in thesearch engine. The solution install Spider Session Remover =============================== This uses Apache mod_rewite to look for specific spiders, and remove the session (osCsid) from the URL, and return a '301' back to the spider. Basically, if the spider tries to do this: GET /www.example.com/product_info.php?products_id=24&osCsid=ac8d8926059625ecb8dd9115f91d5f8a the Apache mod_rewrite will rewrite the url to be: GET /www.example.com/product_info.php?products_id=24 and also return a "301" (Moved Permanently) to the spider. So over time all your duplicatd content is removed.
0 items
Share Product
Share via E-Mail Share on Facebook Share on Twitter Share on Google Buzz Share on Digg
Quick Find
Use keywords to find the product you are looking for.
Advanced Search