The Base Config 090320

[lnminente]

A new version of the base config for proxomitron, most of their filters where rewritten. New vars and flags introduced to make it easier to write your own filters.

It is my work from august/08 of learning and testing filters with the intention of improving and making easy for all to write filters, also to hope nobody has to waste his time again doing this boring thing. when i decided to create my own config and i found it was really complicated to start it from scratch.

So i decided to write a base config, where all people could start writing their own configs but not starting from a blank file. It was born for the need of having different personal configs but sharing some basics to make it easy sharing filters.
It's only a start, people could follow it or not, but it will be here, and will help a lot to capture new talented people to our proxoworld.

I wrote it trying to get the easiest way to understand how it works only by reading the titles of the filters, so the help file would be the minimal, in fact there is no help file. In the next lines i will explain the basics from this config, any question about this config will be answered here. For other questions just go the the help file of proxomitron or ask in the forum, i would like you to remember that.

What is the base config? ___________________________________
It started here:

(Dec. 17, 2008 05:06 PM)lnminente Wrote:  

What about writing a new config between all creating a new standard?


It should modify almost nothing, it is the base.

It should decide what must be or not parsed by proxomitron (many files are not filtered by default. For example the syndication of this forum)

It should define many variables, wich would act to give more safety to our filters, making them also more easy to write. (Defining variables to know where we are, html, body, head, post-html, script, url parts, etc...)

We could investigate to create a good method to filter things but using Header filters, injecting info in the URL to know if it is offsite, to write some info in the killed page, etc...



The basic filters everybody should share and know how to use. With that, we could create complex filters and share them easily in the forum.

Filtering ads it's nice but not to be included in the base. Althought we could create a .js file with standard functions like toggle.

It might seem a very a ambitious project, but it isn't really. Its only to start it. Some people here, me one of them, wrote their own configs from scratch and could help a lot with our experiences. It could be like a Sidki lite, but improved with new helps.


I hope everybody get the idea behind this thread, and think the same. From "the base config" different configs could be created easily.

Write what you think, what would you add, from where you want we begin...

Quoted from: The base config


Lets begin____________________________________________________

1) It uses port 8080, configure your browser to use it, search in the forum. And by now it not filter SSL pages.

2) All the filters in the base config have a "·" in his titles to do easier updates. If you didn't used "·" in your filters write "·" in the find bar in web filters and in header filters and delete all the filters you see. Save the config as "MyFilters.cfg".

3) The URL title format for the header filters will be:
(URL or !)(space):(Zero for outcoming, I por incoming)_(#section).(#filter).(#subfilter/related filter) (Title) {autors}(ISO 8601 Calendar date: YYMMDD)
Example1: URL :0_0.3.1 Outcoming example {sd,th}040526
Example2: URL :0_I.2.2 Incoming example {ln}090125
Example3: ! :0_2 Set vars for outgoing conns {ln}090317·
The "! :" will be parsed before than the "URL :" filters.

4) There is a flag "Set Debug level" wich can be enabled by one header filter since ougoing or incoming in headers, or since webfiltering. Just go to the header or web filters window, at botton there is a find bar, write there "debug", and you will find the filter easier.

5) The expression to log in the log window when debug mode is enabled is

Code:

$TST(keyword=*.i_level\:[5].*)$LOG(C Text to log)

6) Format for logging in the log window by the header filters is

Code:

$LOG(C$DTM(c),"Number and title of the filter" etc...)
example: $LOG(R$DTM(c),I_3.1 Kill error 404 Not Found: $GET(uHost)...$GET(uFile).$GET(uExt))

7) Colors for logging in the log window:
Red for killing, Cyan for debugging, Gray for not important things, Violet for other things.
Others in specifical filters(Blue for setting debug, Green for some outgoing/requests, Yellow for some incomming)

8) Format for logging at 090318 is

Code:

$ADDLST(debug,title\t$DTM(Y/M/D H:m:s't)\t$GET(uDom)\t$GET(uExt)\t$GET(ContentType)\t$GET(Resp)\t$GET(ContentLength)\t\u\t\9\t\1\t\3)

To have the latest version see it at the web filter "Common code" (Write it in the find bar)

9) Define parts of html section or "Html zone filters 2" Fully rewritten
Now the zones will be defined by only one var called "zone". It works in the following way:
First three areas: before, inside and after of <html>*</html>
- <start> of HTML file (before), zone=.PreHtml.
- <html> tag (inside), zone=.Html.
- </html> tag (after), zone=.PostHtml.
Later, depending if we are in PreHtml or in Html zone and we find a...
- <head> tag, zone=.(PreHtml|Html).head
- <body> tag, zone=.(PreHtml|Html).body
You know an html page use to be (start of file)...<html>...<head>...</head><body>...</body></html>...(end of file). It's easy, no? Read it again slower if you didn't caught it.

10) Parts of the web filters
.1- USER FILTERS 1, and USER FILTERS 2. These sections will be for your own filters. Use always the section 2, unless it doesn't work and must be placed in section 1.
.2- FILTERS FROM DEVELOPERS. This section will be used for people wich make public their filters, since now we will call them filter packs. An filter pack for ads from kye-u, a filter pack for ad-containers from z12, etc...
.3- CODE EXAMPLES AND UTILITIES. This section is for showing some important and useful expressions for proxomitron.

11) Variables used in the Base config:
- [filterable] values: 1=webfiltering enabled, (empty)=webfiltering disabled
This var is used to know if proxomitron will web filter a connection. At the beginning it is defined as soon as the server respond us telling the content-type ("Set vars for incoming conns 2") knowing what connections are filtered by default by proxomitron (Thanks Sidki for documenting it). Later each time we use the command $Filter we must remember to update this var.
Example 2: We write a filter to enable web filtering, we must use (^$TST(filterable=*))$FILTER(true)$SET(filterable=1)
If we write a filter to disable web filtering, just use the opposite, it would be $TST(filterable=*)$FILTER(false)$SET(filterable=)

- [imgExt] values: 1=image extension, (empty)=non image extension
This var is used to modify fastly the comportament of other filters depending if we are downloading a file whose extension is the same than a typical image. It is defined at "Set vars for outgoing conns". A trick to find it fastly is opening the config with a text editor and search for "$SET(imgExt=". This trick can be used for all the variables, but others will be set in diferent filters, like "filterable" for example.

- [keyword] values: 5=debug mode
It is the only value wich has by now. Used the 5 to stay more compatible with filters used in the Sidki config.

- [ContentType] Its value is all the words before a ; in the incoming HTTP header Content-Type

- [ContentLength] Its value is the one from the incoming HTTP header Content-Length

- [Resp] Defined with the same falue than the incoming HTTP header Resp

- [tScript,tAnc,tStyle]
What some of us call the "inside vars". They will be defined by this header filter and by the last web filter from the list. There are three tScript,tStyle and tAnc. Their value will be 1 when proxomitron is parsing a script, a style or an anchor respectively, and empty when parsing outside of them.
So to prevent a string matching inside a script, style, etc we will use

Code:

(^$TST(tScript=*)) or (^$TST(tStyle=*))

note: when importing filters from sidki config remember it uses "script" instead of "tScript". For easy remembering, I decided to use a "t" at the beginning of all the inside vars.

This config uses the URL-Parser list, so it uses the variables uProt, uHost, uPort, uPath, uFile, uExt and uDom. To know about them, go to the URL-Parser list. Later there is a filter wich updates uExt and uFile when there is an incoming header Content-Disposition.


Explaining some filters___________________________________

- Test order of header filters. It should never match, if it match is because there is another filter setting the var keyword. This could happen when proxomitron saved the filter disordered because you introduced a header filter with a wrong name format. Remember to add a space to the url filters.
Good: "URL :"
Bad: "URL:"

- Redirect to debug, also Redirect to bypassed source. These header filters moves you fast to the corresponding functions of proxomitron if you press F6 or F8 while starting to load the page.

- Add to Protect list. Thanks to this header filter, you can click in a link to add entries to a list. To use it in a web filter, add an href with an address wich would be matched by this header filter.
At 090320 it is "http://addtolist//\1~reload~\2" where \1 is the address to add to the list, and \2 is the page will be loaded after closing the dialog box, normally will be \u.


Other_________________________________________

How do i install the base config set? Copy the zip file to the folder of proxomitron, and unzip his content there.

Finding fails to a config can be easy, help us to fix them!

Since this moment this config is public without any warranty or responsability from me, it could kill your cat and give live to your computer. This is lazylessware, it is free for non comercial use and for non lazy people.

Dedicated to my family for letting me write this. Special thanks goes to Sidki3003 for his config and for documenting many things about proxo, to JD5000 because i learned the proxomitron code reading his configs, to Kye-U for maintaining and improving this forum, and to all the people who use to login here sharing nice info and good conversations. Z12 hope you get better soon.

The Base config concept is started, now it wants you, what could you do for it?

[Siamesecat]

I just looked at the cfg file from the base config zip file, and I have some questions about it.
What are extrange characters? I could not find a definition for the word extrange.
Why do the names of some of the filters have a long string of "N"s?
Why would you want to log incoming images?
What does the filter to log incoming non images do? Are you really going to log anything that is not an image?
There is a header filter called "Add to protect list". From what would you be protecting the browser? What sort of things would you put in there?

[lnminente]

Hi Siamesecat,
-Strange characters in my own english means Odd chars hehehe, i will change it, thanks!

-The filters with a lot of NNNNN are only separators, with the open window you will see they have no code, only (^?) or text.

- There are 4 filters, for loging request or incoming connections, and for images or for non images. All of them are intented to have the log window opened with HTTP headers loging unticked, replacing them with only the most important info. The filter to log requesting non images, wich is the only one enabled by default, will write in the log window a green line with the most important request we would need to see. We will see there when we are downloading a video, a sound, a javascript, a flash, etc
In Log requests, the condition is $TST(imgExt=*), to see what extensions are intended to be images in the actual or a future base config open it with a text editor and search for "$SET(imgExt=". In this version, you will find $TST(uExt=png|gif|ico|jpg)$SET(imgExt=1)
I only added the typical web image extensions.

The filter to "Log incoming images" is disabled by default it is ther only as complementary to the "log non images". They work having in consideration the imgExt and if the incoming http header Content-Type is equal to image/*. Both of the "log incoming" filters are useful for debugging purposes when we are writing a header filter for example to block some requests.

Of course i will log anything that is not an image, the non images are the most important to me, maybe you thought log was "log to a file". BTW to mean "log to a file" i write "logF" in the title of the filter.

-The protect list is a header filter needed for my ad filters to give more confort to add exceptions. All the matched webfilters for ad links, flash objects, iframes, scripts, will have two links, the first for toggle and the second to add to the protect list.

NOTE: I will be updating the first post, with the questions/answers from this post

[sidki3003]

lnminente, could you place your required local.ptron files in a subdirectory too next time?

Because, since i always have at least a dozen config sets installed, adding your set - right now - involves mass-replacing "http://local.ptron/" in the respective files.

[ProxRocks]

i second that emotion...

(1967 hit single by The Miracles)

[lnminente]

Of course, it's very logic. Count with it

Edit: Also i will move my own ViewSrc.css to a folder named "Extra", to avoid some overwrites.

[ProxRocks]

lnminente, how's progress on the "base config", haven't heard anything in a while...

[lnminente]

Thanks for asking PR, it goes really well, really stable and ready and easy to receive new keywords and filters. Lot of things have been introduced since the 090320. I tried to release it some weeks ago, but i didn't publish it because it needs a lot of work to write the help file, and everytime i write the help file i see new things wich could be interconnected, filters wich can be splitted to share functionality, etc... Maybe should be better to release it as it is now, and later change some small things.

Very probably the next one will be my last release, as a base i see very few things i would add to it. It doesn't mean i will abbandon the base, it means it is near all i need from it

Probably i would release it together with a basic filter pack to block most ads and some other useful filters to make it useable for everyday. Soon i hope...