Webtrends    forums.webtrends.com    Webtrends Online Forums  Hop To Forum Categories  Archives - Older Forum Posts and Product Lines  Hop To Forums  Reporting Center    msnbot and NaverBot not being filtered out??
Go
New
Find
Notify
Tools
Reply
  
-star Rating Rate It!  Login/Join 
<Anonymous>
Posted
I have the entries "msnbot" and "NaverBot" within my list of spiders and crawlers to exclude but they are still being listed within the reports. Has anyone had any luck filtering these two out? Thanks.
 
Reply With QuoteReport This Post
<Anonymous>
Posted
Not sure if this is related, I noticed after adding quite a few new robots to our list that MSNBOT as well as some others were no longer being filtered out as well.

Seems to be the list can only be a certain length for each exclusion filter or it will not process them all.

I made another exclude filter for the spiders and now they are getting filtered out again.

Hope this helps.
Jim
 
Reply With QuoteReport This Post
<Anonymous>
Posted
That worked. Looks like our list was too long. Thanks.

>Not sure if this is related, I
>noticed after adding quite a
>few new robots to our list
>that MSNBOT as well as some
>others were no longer being
>filtered out as well.
>
>Seems to be the list can only
>be a certain length for each
>exclusion filter or it will
>not process them all.
>
>I made another exclude filter
>for the spiders and now they
>are getting filtered out
>again.
>
>Hope this helps.
>Jim
 
Reply With QuoteReport This Post
Posted Hide Post
Further to this post, I was wondering about these crawlers that aren't being filtered using Web Trends' native 'Exclude crawler' filter.
Today I found some whackiness in my reports and my investigation revealed a crawler called HTTRACK which was not being excluded from the report.
How is one supposed to identify that this is an issue ? Does it have to take someone reporting to me that the numbers look wierd ? Is it possible to tell which crawlers are excluded in the native 'Crawler' list ?
 
Posts: 4 | Registered: October 10, 2005Reply With QuoteReport This Post
Posted Hide Post
The native "crawler" list is viewable, yes.

And if you're in charge of WebTrends, it's up to you to create and check certain reports to keep up with the crawlers that have targeted your site. It's part of WebTrends management, since WebTrends does not issue frequent updates of this file. They would have to be doing it weekly, at least, to keep up. It would be nice if WT did it, but it only takes about 1-5 minutes a week at most to do it.


Chris G
 
Posts: 2193 | Location: Ann Arbor, Michigan | Registered: July 13, 2005Reply With QuoteReport This Post
  Powered by Social Strata  
 

Webtrends    forums.webtrends.com    Webtrends Online Forums  Hop To Forum Categories  Archives - Older Forum Posts and Product Lines  Hop To Forums  Reporting Center    msnbot and NaverBot not being filtered out??

© 2009 Webtrends Inc. All Rights Reserved.