Viewer for $ADDLST logs?

[sidki3003]

Hi all,

I'm setting up my filters to use this terrific new $ADDLST command,
so that they write any matches to a logfile.
This works great and looks like this:

[2002-6-16 14:58:41] HDR CT_plain text/plain http://developer.apple.com/internet/javascript/internetdev-sniffer.txt
[2002-6-16 19:35:40] WEB Banner Add Me! http://66.31.61.166/flogviewer/mainmenu.asp
[2002-6-16 20:04:01] WEB Flash "img/movie.swf" http://www.esecurity.co.kr/html_eng/products/es_ms_02.htm

Now i wonder whether someone knows a log viewer, that can do column sorting,
so that i can see for instance which filters are never triggered, which sites are most aggressive, ...

regards, sidki


 

[Arne]

I think it would be easier if you inserted som kind of a delimiter like a comma, tab or semicolon between the fields. Now it is only one long string for each input, which is difficult to separate.

Best wishes
Arne
Imici username= Arne

[dave1006]

I think whitespace can be used as a seperator.

As for this function of recording 'hits', i've always felt this would be a great feature within Proxomitron, if it were added. Just a little counter for each filter with a start date - so you can see hits (total/per day etc).

-------------------------
|David Gallagher        
|dave at smokeajay.co.uk
-------------------------

[sidki3003]

You are right Arne. For now i inserted a tab since it's easier for manual reading.

 

[Arne]

About whitespaces. They could be used, but in this case there are a bit too many of them:
...] WEB Banner Add Me! http.......
So one of the other will be easier to deal with.

And when it is delimited, one can even copy paste directly into a spread sheet and sort it there, if you don't have a flat database viewer program.

Best wishes
Arne
Imici username= Arne

[sidki3003]

Didn't see you post Dave. The $ADDLST command can also just count hits

sidki


 

[dave1006]

hi,

yeah Arne using a seperator is the simplest way, but you can use whitespace, as we know the first two 'tokens' are the date, and the last token is the link, so using a tiny bit of math you could count all other tokens as the filter name match

hi sidki,
Does the list count up though, or just log a certain number of entries?
And yeah, I think the addlist feature is very useful for logging etc, but personaly i'd still like a tiny counter within proxo itself

If there was a wishlist, that'd be my 'thing'

-------------------------
|David Gallagher        
|dave at smokeajay.co.uk
-------------------------

[sidki3003]

The list would grow one byte on every hit, so you'd see it by the filesize.
Not so comfortable as a real counter i admit.


 

[dave1006]

I spose you could use windows explorer and set the 'order by' to filesize' and, presuming you have a list/file for each filter,by filter name, you'd then see the filters names in order of 'hits'

-------------------------
|David Gallagher        
|dave at smokeajay.co.uk
-------------------------

[sidki3003]

Just realized: The only two things missing to create a CLF log with proxo is catching the outgoing HTTP commands (GET, POST, ...) and the incoming HTTP/1.x lines.

 

[sidki3003]

I found a viewer
It's not very pretty and accepts only tabs and commas as delimiter, but it does its job (column sorting, nothing more). Kiwi Logfile Viewer:
http://www.kiwisyslog.com/software_downloads.htm

BTW, i use their syslog daemon (for firewall and NT log messages). It's quite nice.

sidki


 

[Arne]

Could you post an example filter that writes to this file, so that others can see how you put the date and other info in there.?

Best wishes
Arne
Imici username= Arne

[sidki3003]

Sure

[Blocklists]
List.LOGfile = "..Log.txt"

[Patterns]
Name = "Kill: Flash animations (no ACR)"
Active = TRUE
URL = "($TYPE(htm)|$TYPE(js))"
Bounds = "<object*</object>|<embed*>( </embed>|)"
Limit = 1024
Match = "[^>]++(codebase|type)=$AV(*(flash|shockwave)*)*"
        "&(*<param ( name=$AV(movie|src)| value=$AVQ(1))+{2}|*src=$AVQ(1))"
        "&$ADDLST(LOGfile,[$DTM(d T)]   WEB Flash   1    u)"
Replace = "<span class=prox style=display:none;><a href=1 target="_top">[flash]</a></span>"


sidki


 

[Arne]

Thanks, I wa thinking of adding something to the P-FAQ tomorrow, and then it was nice to have something with the date and Tab in it as well

Best wishes
Arne
Imici username= Arne

[Scott Lemmon]

Please - ADDLST should not be used as a make-shift log file.  It's very likely to cause problems and eat up memory. Remember, it's parsing everything you give it as a matching command.