Discussion:
Crosspost Filter Help Agent 2.0
(too old to reply)
Fasa
20 years ago
Permalink
Some newgroups are under a hugespam attack, and the headers and other
identifiers are randomly generated and gibberish--no meaningful
content. So the regular filters don't stop it.

I read somewhere once where you couild set agent up to simply kill all
crossposted messages completely, or you could tell it to allow those
only posted to two or less groups, etc. In other words, when
dowloading messages, Agent would look in all the headers and see which
messages were posted to more than one, two, etc. groups, and jsut kill
them completely.

Can anyone tell me how to do that?

and thanks a lot for any repsonses.
Scribbles
20 years ago
Permalink
On Wed, 01 Jun 2005 17:06:48 -0700
...
A similar issue was discussed in late April or early May of this year. You may
be able to find the thread on Google. Unfortunately, I don't remember the
thread title.

A clever person in that thread suggested the following which I now use.
Certain "nuts"cross post phlegm to several groups, one of which I want to read
without the "nuts". One of the cross-posted "nut" groups happens to be
alphabetically before the group I want to read. That is key to this method.
There is a 3rd party program to reorder the groups if necessary.

I have set the "nut" group properties to "not retrieve messages longer than 0
lines" This gets only that group's headers for agent's cross-posting
management to do its work on. Agent then goes on to get the new message bodies
in the other groups I want to read without the "nuts".

The "nut" group may show a hundred new messages available for downloading. I
press 2, to go into the message titles pane; Ctrl A, to select all the
messages; I and I again, to ignore all the messages in the "nut" group.

It has worked well for me. Good luck.
Dave Fawthrop
20 years ago
Permalink
On Wed, 01 Jun 2005 17:06:48 -0700, Fasa <***@seop.com> wrote:

|
| Some newgroups are under a hugespam attack, and the headers and other
| identifiers are randomly generated and gibberish--no meaningful
| content. So the regular filters don't stop it.
|
| I read somewhere once where you couild set agent up to simply kill all
| crossposted messages completely, or you could tell it to allow those
| only posted to two or less groups, etc. In other words, when
| dowloading messages, Agent would look in all the headers and see which
| messages were posted to more than one, two, etc. groups, and jsut kill
| them completely.
|
| Can anyone tell me how to do that?


Nfilter http://www.nfilter.org/ will help, not sure about Windoze XP

Also see sig
--
Dave Fawthrop <dave hyphenologist co uk>
Killfile and Anti Troll FAQs at
http://www.hyphenologist.co.uk/killfile.
Fasa
20 years ago
Permalink
I'm nut really sure how the first response for 'nut' groups works, but
the nfilter seems like an option unless there is a way to do waht I
want in Agent. Some guy wrote a while ago about modifying one of
agents files to help filter. It was easy really. What he did was jsut
tell Agent not to download any files or delete afterwards any
messages taht were crossposted. He limted the crossposting filter to 2
groups, which stripped out the spammer--since he was crossposting to
5+ at a time.

Is there a way to do that in Agent itself by modifying a file?




On Thu, 02 Jun 2005 07:04:28 +0100, Dave Fawthrop
...
Fasa
20 years ago
Permalink
I forgot to add that regular filtering won't work on mesage ID and
subject because each one is randomly generated spoof. So the only way
to filter this crap is to just deny any message that is crossposted.
Is there a way to tell Agent to do that?
Ralph Fox
20 years ago
Permalink
On Fri, 03 Jun 2005 17:17:17 -0700, in message
Post by Fasa
I forgot to add that regular filtering won't work on mesage ID and
subject because each one is randomly generated spoof. So the only way
to filter this crap is to just deny any message that is crossposted.
Is there a way to tell Agent to do that?
Read Scribbles's reply to you.

Agent's crosspost filtering will filter a crossposted message from
the second and subsequent groups in which you get the crossposted
message header, but not from the first group.

Agent cannot filter a crossposted message header from the first
group in which it appears.

You can still make use Agent's of crosspost filtering to completely
kill many crossposted message headers. Scribbles has told you how.
--
Cheers,
Ralph

"Spoon-feeding in the long run teaches us nothing but the shape of the spoon." -- E. M. Forster
Ralph Fox
20 years ago
Permalink
On Fri, 03 Jun 2005 16:47:12 -0700, in message
Post by Fasa
I'm nut really sure how the first response for 'nut' groups works, but
the nfilter seems like an option unless there is a way to do waht I
want in Agent. Some guy wrote a while ago about modifying one of
agents files to help filter. It was easy really.
Have you searched Google Groups for that message or the thread?
Post by Fasa
What he did was jsut
tell Agent not to download any files or delete afterwards any
messages taht were crossposted.
If "delete afterwards" means after the body is downloaded,
then there is the following option.

If you use Agent's global search to search for

Newsgroups: {,}

you will find all messages with bodies where the message has been
cross-posted to 2 or more groups. You can then delete these
messages, a group at a time.

Note that this will not find cross-posted messages where you only
have the header, and not the message body.
Post by Fasa
He limted the crossposting filter to 2
groups, which stripped out the spammer--since he was crossposting to
5+ at a time.
Is there a way to do that in Agent itself by modifying a file?
Anything like this is possible with enough modification -- the operative
word being "enough".
--
Cheers,
Ralph

How To Ask Questions The Smart Way: http://www.catb.org/~esr/faqs/smart-questions.html#before
Fasa
20 years ago
Permalink
Good old Ralph! I remember you from years ago helping me with a
similar problem. Thanks a lot man!

I got the nfilter program and installed it. So that should allow me to
filter by # of xposts. However, I'd like to discuss this a bit
further.

(1) How do I search for the "nut" article? What "criteria?"
(2) So there is no way for Agent to see the header xpost information
without first getting the body first?
(3) So what is the prognosis for Agent to incorporate some spam
filters we can use, much like nfilter?

On Sat, 04 Jun 2005 00:47:35 +0000, Ralph Fox
...
Fasa
20 years ago
Permalink
OK Ralph.

I set up Nfilter and got it working. Two questions:
(1) is message retrival slower than without using nfilter?
(2) I set up the filter like this

* drop xref:*:*:*:*

To drop xposts greater than 3, jsut asw it says in teh nfilter help
file:

3.6 Can I filter out excessive cross-posts?

This can be done in a slightly roundabout way. You have to count the
commas in the newsgroups line. It is more efficient to use the Xref:
header however and you can filter crossposts on this header by
counting colons. For example:
* drop xref:*:*:*:*
This filter will drop any posts with three or more newsgroups in the
XRef line.

The problem is I get the error message:
FCF:0B54 +086 Ignoring bad entry 'filter.dat' line : 46 =

So what am I doing wrong here?
...
Ralph Fox
20 years ago
Permalink
On Sun, 05 Jun 2005 18:16:14 -0700, in message
Post by Fasa
I set up Nfilter and got it working.
(1) is message retrival slower than without using nfilter?
If you are filtering on headers which are not in your server's overview,
or if you have a lot of filters, then it could be slower.
Post by Fasa
(2) I set up the filter like this
* drop xref:*:*:*:*
To drop xposts greater than 3, jsut asw it says in teh nfilter help
3.6 Can I filter out excessive cross-posts?
This can be done in a slightly roundabout way. You have to count the
header however and you can filter crossposts on this header by
* drop xref:*:*:*:*
This filter will drop any posts with three or more newsgroups in the
XRef line.
FCF:0B54 +086 Ignoring bad entry 'filter.dat' line : 46 =
So what am I doing wrong here?
Do you have regular expression filtering turned off or turned on?

That example filter is not a regular expression filter.

If you have regular expression filtering turned on, try this

xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)
--
Cheers,
Ralph

"A lack of planning on your part does not constitute an emergency on mine."
Fasa
20 years ago
Permalink
Regular expressions was turned off Ralph. I tried it on and off and
got the same error. Also, I tried your expression after turning on
"regular expressions" and didn't get the error. So the expression you
gave me

xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)

does it filter xposts greater than 3, 2, ? Just need that variable so
I can see what it's doing. I assume it is 3 by the way the pattern
looks.

Also, why do you think the help file gives an invalid expression to
use?

Thanks again .

.
On Mon, 06 Jun 2005 05:15:45 +0000, Ralph Fox
...
Fasa
20 years ago
Permalink
Ralph, also on teh speed. It's REALLy slow, like to download 160, 000
headers without Nfilter, it's about 3 minutes, With Nfilter, maybe 2
hours.

Is there a setting I need to change that may be slowing it down this
much?

On Mon, 06 Jun 2005 05:15:45 +0000, Ralph Fox
...
Fasa
20 years ago
Permalink
Ralph,

After restarting nfilter I'm getting errors again:

(,0) FCF:04EC +210 Ignoring bad entry 'filter.dat' line : 46 =
(,0) FCF:04EC Ignoring bad entry 'filter.dat' line : 47 =
xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)

This is my filter dat.:

#
# NewsProxy sample filter file
# ============================
#
# This is a sample filter file with comments.
#
# Note that lines that begin with a # are ignored.
#
# $Workfile: newsproxy.ini $
# $Archive: /NewsProxy/newsproxy.ini $
# $Date: 11/03/99 13:26 $
# $Revision: 1 $
# $Author: Mark Burkley $
# $Log: /NewsProxy/newsproxy.ini $
#
# 1 11/03/99 13:26 Mark Burkley
# Initial version
#
# ============================
#
# Flag any posts in alt.animals.foo which are cross-posted to
# newsgroup alt.fan.foo as JUNK
#
#alt.animals.foo flag:JUNK xref:*alt.fan.foo*
#
# Flag any posts in any group starting with alt.disasters which are
# cross-posted to any newsgroups that contain alt.aviation as XPOST
#
#alt.disasters.* flag:XPOST xref:*alt.aviation*
#
# Drop any posts in any newsgroups that don't have an
nntp-posting-host
# header and that are cross-posted to a newsgroup that contains the
# word flame.
#
#* drop nntp-posting-host:(none)
xref:*flame*
#
# Flag any posts in any newsgroups that passed through the news site
# news.foo.com as FOO
#
#* flag:FOO path:*news.foo.com*
#
# Drop any posts from ***@domain that are posted to a rec
# group.
#
#rec.* drop from:****@domain*

xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)




On Mon, 06 Jun 2005 05:15:45 +0000, Ralph Fox
...
T-Bone
20 years ago
Permalink
Look closely at what it's telling you. Lines 46 & 47 are erroneous. Look
at your nfilter.dat: line 46 is blank, line 47 has improper syntax.

Line 46 is a blank line -- Blank lines are not allowed, remove line 46.

Line 47 needs the newsgroup and action

Proper syntax is: newsgroup action[:marker] header:value [header:value]

Here's an example that will drop all matching headers from every group:
* drop xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)
Post by Fasa
Ralph,
(,0) FCF:04EC +210 Ignoring bad entry 'filter.dat' line : 46 =
(,0) FCF:04EC Ignoring bad entry 'filter.dat' line : 47 =
xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)
Fasa
20 years ago
Permalink
OK I fixed the blank lines by taking out all entries except the one
below. Ralph gave me this

xref:(.+:[1-9]+.+:[1-9]+.+:[1-9]+.+)

by request that I assume will drop all headers x-posted to more than 3
newsgroups.

Is that right?

It semms to work now as I am getting no errors.
...
Fasa
20 years ago
Permalink
OK no errors, but the xref lines in the sporgeries are still showing
multiple xposting, whcih means the filter isn't working.
...
Ralph Fox
20 years ago
Permalink
On Sun, 05 Jun 2005 17:43:54 -0700, in message
Post by Fasa
I got the nfilter program and installed it. So that should allow me to
filter by # of xposts. However, I'd like to discuss this a bit
further.
(1) How do I search for the "nut" article? What "criteria?"
Not for the 'nut' article.

Search for the article which you described in news:***@4ax.com

| Some guy wrote a while ago about modifying one of
| agents files to help filter. It was easy really. What he did was jsut
| tell Agent not to download any files or delete afterwards any
| messages taht were crossposted. He limted the crossposting filter to 2
| groups, which stripped out the spammer--since he was crossposting to
| 5+ at a time.

You would know what to "criteria" search for, better than I would.
I don't recall that article.


As for the 'nut' article which Scribbles mentioned, try this search
http://groups-beta.google.com/groups?as_q=agent-group-order+cross-post&as_ugroup=alt.usenet.offline-reader.forte-agent&as_scoring=d
Post by Fasa
(2) So there is no way for Agent to see the header xpost information
without first getting the body first?
Not at present.
Post by Fasa
(3) So what is the prognosis for Agent to incorporate some spam
filters we can use, much like nfilter?
I don't work for Forté so anything I might say is just speculation.

However, you can draw some alternative conclusions by thinking
hard enough about what such a change would entail -- such as will
any existing functionality be lost and will the database file
format have to be redesigned.
--
Cheers
Ralph

Which is worse, ignorance or apathy? Who knows? Who cares?
Fasa
20 years ago
Permalink
Nfilter--no longer works on Windows XP platforms. Won't intall.

On Thu, 02 Jun 2005 07:04:28 +0100, Dave Fawthrop
Post by Dave Fawthrop
Nfilter
Ralph Fox
20 years ago
Permalink
On Fri, 03 Jun 2005 17:24:38 -0700, in message
Post by Fasa
Nfilter--no longer works on Windows XP platforms. Won't intall.
You have a bad download.

You downloaded your copy of NFilter from http://www.nfilter.org/ .
That server is deliverying binary files 1 byte shorter than the file should be.
XP will not run a .exe file that is 1 byte short.

Nfilter v1.20 installer np-120.exe

good download size...... 629,736 bytes
bad download size....... 629,735 bytes

Go download a good copy from http://www.netaxs.com/home/v/nfilter/
--
Cheers,
Ralph

Seek and ye shall find,
Ask and ye shall be flamed.
unknown
20 years ago
Permalink
Ralph Fox <***@xtraxtra.coco.nznz.undo-echo.invalid> writes:

| Go download a good copy from http://www.netaxs.com/home/v/nfilter/

This URL is a no-go for me, Ralph. Would you confirm this please.

Fordhook
m***@gmail.com
20 years ago
Permalink
On Sat, 04 Jun 2005 12:30:49 -0400,
][v][Førdhøøk][v][ wrote:

How did you create the 'o's with slashes in your name?
Ralph Fox
20 years ago
Permalink
On Sat, 04 Jun 2005 22:42:38 GMT, in message
Post by m***@gmail.com
On Sat, 04 Jun 2005 12:30:49 -0400,
How did you create the 'o's with slashes in your name?
There are several ways, including..

1. Start -> Programs -> Accessories -> System Tools -> Character Map

2. Hold down the left Alt key, and keep it down while typeing 0248

3. Use a Danish keyboard or a Norwegian keyboard, both of which
have Ø and ø keys on them.
--
Cheers,
Ralph

How To Ask Questions The Smart Way: http://www.catb.org/~esr/faqs/smart-questions.html#before
unknown
20 years ago
Permalink
Ralph Fox <***@xtraxtra.coco.nznz.undo-echo.invalid> writes:

| On Sat, 04 Jun 2005 22:42:38 GMT, in message
|  <***@4ax.com>, ***@gmail.com wrote:
|
| > On Sat, 04 Jun 2005 12:30:49 -0400,
| > ][v][Førdhøøk][v][ wrote:
| >
| > How did you create the 'o's with slashes in your name?
|
|
| There are several ways, including..
|
| 1. Start -> Programs -> Accessories -> System Tools -> Character Map
|
| 2. Hold down the left Alt key, and keep it down while typeing 0248
|
| 3. Use a Danish keyboard or a Norwegian keyboard, both of which
| have Ø and ø keys on them.

4. Copy and paste. :)

Thanks for the location of nfilter.

Fordhook
Ralph Fox
20 years ago
Permalink
On Sat, 04 Jun 2005 20:13:34 -0400, in message
Post by unknown
Thanks for the location of nfilter.
You're welcome.

Thanks also to those who have mirrors _without_ a robots.txt exclusion.
Ralph Fox
20 years ago
Permalink
On Sat, 04 Jun 2005 12:30:49 -0400, in message
Post by unknown
| Go download a good copy from http://www.netaxs.com/home/v/nfilter/
This URL is a no-go for me, Ralph. Would you confirm this please.
It seems to have vanished since I last looked at it.

At one time I had been able to get old copies from the wayback machine's
archive of Mark Burkley's site which used to be at www.nfilter.org.

Very interestingly, the current owner of the www.nfilter.org domain
has set a robots.txt exclusion on on 26 August 2004, which prevents
the wayback machine from providing even copies archived from the
previous owner (Mark Burkley) when there was no robots.txt exclusion.
http://web.archive.org/collections/web/faqs.html#exclusions


No probs -- there is another mirror of Mark Burkley's original
site at http://li.ve.ly/nfilter/ .
Loading...