Post Reply 
EasyList to Privoxy Syntax. A few things to bear in mind…
Jan. 20, 2016, 11:27 PM (This post was last modified: Dec. 08, 2017 10:30 PM by Faxopita.)
Post: #1
EasyList to Privoxy Syntax. A few things to bear in mind…
A few paragraphs below, syntax converter means either privoxy-adblock script for OS X or adblock2privoxy package for Linux. This discussion is mainly for privoxy-adblock as I did not test the Linux version, but you can check whether your own converted files fall into this category as well or not.

This post is a copy of what I already posted at GitHub: “Syntax Error in Resulting .action”.

What I realised is that converting EasyList filters into Privoxy syntax is not that straightforward. Sure, the syntax converter deals correctly with the template format used by Privoxy for custom filters and actions, but does not output the correct syntax for most of the entries found in the resulting .action files provided by the syntax converter.

Also, if my understanding is right, EasyList supposes that Adblock Plus does not make any difference between a domain or path in terms of string matching. On the other hand, Privoxy does make a difference between the domain and the path. One notable example: if it’s a path, Adblock Plus doesn’t care whether string matching targets the root folder of the site or any location further down the tree. Privoxy does care. In Privoxy, if an entry does not start with a slash (/), then the entry is deemed as part of the domain name.

Based on those considerations, I created an “add-on” to privoxy-adblock to make the resulting .action files perfectly compatible with Privoxy. Because Privoxy has two different syntaxes for domain and path (see note at the end of this post), I built the add-on so it deems a malformed entry as domain or path based on a “no non-sense” approach by simply modifying the syntax so that the amended entry is now understood by Privoxy.

---

Example of (wrong) original outputs from privoxy-adblock…

Code:
\.com/ads/
.adserver\.com
&ad_id
\.pop-ads\.
/ads/

Which will become compatible with Privoxy after applying the add-on:

Code:
.com/ads/
.adserver.com
/.*&ad_id
/.*\.pop-ads\.
/(.*/)?ads

Above, the use of /.* or /(.*/)? gives a chance for the pattern to be matched further down the path.

The resulting .action files obtained from the add-on—that is run after privoxy-adblock execution—have been successfully integrated in Privoxy and no syntax error was detected by Privoxy, contrary to the situation where the add-on wouldn’t be applied afterwards. Currently the add-on is called from my “master” script, which consists of running first the privoxy-adblock script on several EasyList source files, then my add-on.

The add-on seems a bit messy (or maybe it really is); the add-on looks like it will cause more trouble to your lists, but it won't. At least on my side, after several checks, the outputs from the add-on are really Privoxy friendly.

Note: the patch assumes you have not compiled Privoxy with the option --enable-extended-host-patterns, which is a “switch” to enable PCRE filtering for hostnames. If you enabled PCRE filtering, then comment out the relevant section of the script; see Host Pattern Format section in the add-on.

Update: According to posts below, only OS X users should use the proposed patch. So, Linux users, be pleased not to use it as you are using a better syntax converter.

Update 2: Something that looks better and more accurate than privoxy-adblock or adblock2privoxy is adblock2privoxy (revisited) and its sister page: easylist-pac-privoxy. Finally, Privoxy can be on par with Adblock?

Attached to this post, all you need to rectify your files…

Last update: March 2017 – More user friendly! Wink

-–—

Minuscule donations are always appreciated…
Code:
BTC --> 34WKogWorDoReJ2MSxw8rTsrGD87VMAPJY
BCH --> 1AXwyMdtMFZktZPvXScC58ESUZXptmjvge
DASH -> XusJsETR6PwDnG4Gde7cvGeRhXzUJFSxtD
ETH --> 0xb829FA99AA9AB31C32590dbc88B837bC5D91453e
ETC --> 0x059F128357331c346Ad2E23F95a4639beC3f0b3a
LTC --> MK7vxk93A1M6HHAYT38W8NPJSb8zANqCia
ZEC --> t1JNCuxdZEWUPBQiAzxZPUMqb4BM87sxs9H
DOGE -> DBPAUuCaez4JYGobAn4RHNNhFXwa9u1W6N
STRAT > SgG6jAHuxQfzW1QBaWyQRVdCdSq514BcyM


Attached File(s)
.zip  EasyL2Priv.zip (Size: 459.29 KB / Downloads: 899)
Add Thank You Quote this message in a reply
[-] The following 4 users say Thank You to Faxopita for this post:
kik0s, oldsod, cattleyavns, v4victory
Jan. 21, 2016, 10:14 PM (This post was last modified: Jan. 21, 2016 10:15 PM by oldsod.)
Post: #2
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
Thank you!
This conversion you have created for Privoxy is looking to be precise and accurate.
Job well done and this is very much appreciated.
Add Thank You Quote this message in a reply
Jan. 22, 2016, 04:19 PM (This post was last modified: Jan. 22, 2016 04:28 PM by Faxopita.)
Post: #3
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
Thanks for your feedback and reading my experience with interest. It would indeed be too bad not to let Privoxy correctly take advantage of EasyList filter lists. I’m happy to have raised some awareness about this issue.
Add Thank You Quote this message in a reply
Jan. 23, 2016, 09:31 AM
Post: #4
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
The scripts do not work for me though but nice to see some working scripts to convert
Add Thank You Quote this message in a reply
Jan. 23, 2016, 11:47 AM (This post was last modified: Jan. 24, 2016 02:18 PM by Faxopita.)
Post: #5
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
The archive only contains privoxy-adblock, which is the OS X version. You will need to retrieve adblock2privoxy for Linux. Unfortunately, I can't help for Windows config. Maybe cattleyavns can help here.

As for the add-on, it should equally work well on Linux, although you might need to change some destination paths. For example, tmp_path defines the path you chose as output for the original syntax converter—preferably not the Privoxy folder, because the add-on would also convert other .action files. On Windows, the only thing that came up to my mind to run the add-on is Cygwin…

Unfortunately, I cannot confirm whether adblock2privoxy (Linux version) outputs the right syntax or not. Also, I have not tested the add-on on already correctly-formatted outputs.
Add Thank You Quote this message in a reply
Jan. 31, 2016, 08:29 PM (This post was last modified: Jan. 31, 2016 08:29 PM by kik0s.)
Post: #6
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
faxtopia can you check if the adblock2privoxy synthax is correct? that would be awesone. if you want i can upload the files Smile!
Add Thank You Quote this message in a reply
Jan. 31, 2016, 10:31 PM
Post: #7
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
Could you please upload the files then? Thank you. I'll have a look at them.
Add Thank You Quote this message in a reply
Jan. 31, 2016, 10:44 PM
Post: #8
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
here they are: https://www.dropbox.com/s/rk8tpf5go8m18zz/ab2p.zip?dl=0

i loaded a lot of filters in there. so its a bit huge Smile!
Add Thank You Quote this message in a reply
Feb. 01, 2016, 12:21 AM (This post was last modified: Feb. 01, 2016 12:24 PM by Faxopita.)
Post: #9
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
I must say syntax is overall very well respected and, as a result, you wouldn't need to use the add-on I suggested. A few exceptions though; there are only three instances where there's some syntax issue: too much backslashes. Especially, no need them prior to the square and curly brackets.
Code:
/(.*/)?\\\.sharesix\\\.com/\..*\[a-zA-Z0-9\]\{4\}/
/(.*/)?\\\.filenuke\\\.com/\..*\[a-zA-Z0-9\]\{4\}/
/(.*/)?http://\[a-zA-Z0-9-\]\+\\\.\[a-z\]\+\\/\..*\[a-zA-Z0-9\]\+/

A few thousands entries have .* as end of line. It's not an issue, but unnecessary. Also, there are some entries where I would use (.*[^0-9])? instead of .*, but it is just a personal choice. It's not an issue. For example, I would replace the entry /.*728x90\.php\? by /(.*[^0-9])?728x90\.php\?

The file ab2p.action contains far too many duplicates and could be dramatically reduced in size… About 20 times bigger than it should be.

As for the filter file, it seems OK. I haven't checked thoroughly each item, but it looks good at first sight.
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to Faxopita for this post:
kik0s
Feb. 01, 2016, 06:46 AM (This post was last modified: Feb. 01, 2016 06:50 AM by kik0s.)
Post: #10
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
thanks a lot. i look into the size issue. i guess its because of duplicates in the lists i added to it. every list also contains the normal easylist. so it would be some kind of a speed up for privoxy if it is redused in size.
Add Thank You Quote this message in a reply
May. 15, 2018, 10:38 AM (This post was last modified: May. 18, 2018 03:26 PM by ryszardzonk.)
Post: #11
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
For adblock2privoxy there is now ebuild ready for those that use Gentoo Linux for easy installation.

However I write with somewhat different issue. Among others the adblock2privoxy failed to process several records. For 10 lists I used it was only 443 records which is quite an achievement. My question is however could from list as this records be passed so they no longer error out?

To bring out few examples for most notorious sites
Quote:ERROR: gadzetomania.pl#?#DIV:-abp-contains(REKLAMA) + DIV > A > DIV > IMG (easylistpolish.txt: 2774) - Record type detection failed
ERROR: gadzetomania.pl#?#DIV:-abp-contains(REKLAMA) + DIV > DIV > IFRAME (easylistpolish.txt: 2775) - Record type detection failed
ERROR: wp.pl#?#DIV + DIV + div:-abp-contains(REKLAMA) + DIV (easylistpolish.txt: 2877) - Record type detection failed
ERROR: wp.pl#?#aside:-abp-contains(Reklama) + DIV (easylistpolish.txt: 2878) - Record type detection failed
ERROR: wp.pl#?#aside:-abp-contains(Reklama) + IFRAME (easylistpolish.txt: 2879) - Record type detection failed
ERROR: wp.pl#?#div:-abp-contains(REKLAMA) + DIV > A (easylistpolish.txt: 2880) - Record type detection failed
ERROR: wp.pl#?#div:-abp-contains(REKLAMA) + DIV > DIV > A (easylistpolish.txt: 2881) - Record type detection failed

EDIT: I gave my best shot https://github.com/essandess/adblock2privoxy/issues/6 to getting those rules which work in adblock and preferably with using adblock2privoxy, but I give up. Hopefully someone someone will have more css knowledge and get those filters translated for this nasty web pages types.
Add Thank You Quote this message in a reply
Jul. 21, 2018, 10:41 PM
Post: #12
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
(May. 15, 2018 10:38 AM)ryszardzonk Wrote:  For adblock2privoxy there is now ebuild ready for those that use Gentoo Linux for easy installation.

However I write with somewhat different issue. Among others the adblock2privoxy failed to process several records. For 10 lists I used it was only 443 records which is quite an achievement. My question is however could from list as this records be passed so they no longer error out?

To bring out few examples for most notorious sites
Quote:ERROR: gadzetomania.pl#?#DIV:-abp-contains(REKLAMA) + DIV > A > DIV > IMG (easylistpolish.txt: 2774) - Record type detection failed
ERROR: gadzetomania.pl#?#DIV:-abp-contains(REKLAMA) + DIV > DIV > IFRAME (easylistpolish.txt: 2775) - Record type detection failed
ERROR: wp.pl#?#DIV + DIV + div:-abp-contains(REKLAMA) + DIV (easylistpolish.txt: 2877) - Record type detection failed
ERROR: wp.pl#?#aside:-abp-contains(Reklama) + DIV (easylistpolish.txt: 2878) - Record type detection failed
ERROR: wp.pl#?#aside:-abp-contains(Reklama) + IFRAME (easylistpolish.txt: 2879) - Record type detection failed
ERROR: wp.pl#?#div:-abp-contains(REKLAMA) + DIV > A (easylistpolish.txt: 2880) - Record type detection failed
ERROR: wp.pl#?#div:-abp-contains(REKLAMA) + DIV > DIV > A (easylistpolish.txt: 2881) - Record type detection failed

EDIT: I gave my best shot https://github.com/essandess/adblock2privoxy/issues/6 to getting those rules which work in adblock and preferably with using adblock2privoxy, but I give up. Hopefully someone someone will have more css knowledge and get those filters translated for this nasty web pages types.

Don't give up, try my solution Big Teeth
http://www.prxbx.com/forums/showthread.p...2#pid19432
Add Thank You Quote this message in a reply
Jul. 23, 2018, 11:00 PM
Post: #13
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
Actually I did not give up then and kept working on this. Most recent scripts http://bigvo.gotdns.ch/ssl/gentoo/net-pr...-23.tar.xz for whole proxy toolchain are all there. The most interesting should be sh scripts located in /net-proxy/adblock2privoxy/files/examples/scripts/ directory of that file Smile!
Add Thank You Quote this message in a reply
Oct. 04, 2020, 02:02 PM (This post was last modified: Oct. 11, 2020 03:38 PM by k4nk3i.)
Post: #14
RE: EasyList to Privoxy Syntax. A few things to bear in mind…
Hi, I came across this interesting thread. I am using Privoxy in combination with two scripts, one that download and install blocklists in Privoxy, and the other one that convert the syntax (the script provided by Faxopita). I am using this on a Raspberry Pi to filter my network. I am not sure if the syntax in the .action files is correct. I would greatly appreciate if someone could have a look at some of the action files that I am using (links below).

Thank you

easylist.script.action

easyprivacy.script.action
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: