Something like “decentraleyes” for Privoxy…

[Faxopita]

To roughly mimic “decentraleyes” Firefox add-on…

More info here or there.

Quote:Decentraleyes is a new open source Firefox add-on that aims to improve your privacy while browsing. It does this by hosting CND resources locally. When your browser makes a request for one of these CDN resources, the request is blocked and you are served up a local version instead.

BestVPN

Quote:Can CDNs track me even though they do not place tracking cookies?
Absolutely. Requests to content delivery networks contain a "Referer" HTTP header (originally a misspelling of referrer) that reveals what page you're visiting. Techniques like IP address tracking and browser fingerprinting can then be used to determine your identity.

GitHub/Synzvato/decentraleyes

This rule allows Privoxy to use local versions instead:

Code:

{ +redirect{[email protected]^(https?://)?[^,%]+\b(angular|backbone|dojo|ember|ext.core|jquery.ui|jquery|modernizr.min|mootools-yui-compressed|prototype|scriptaculous|swfobject|underscore.min)\b[^/]*\.js[^,%]*@http://localhost/~Your_Path_to/CommonJS/$[email protected]} }
   .aspnetcdn.com
   .baidu.com
   .cloudflare.com
   .googleapis.com/(.(?!1\.7\.1/))*$ # Example: with an exception; use remote version 1.7.1 instead: ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js
   .jquery.com
   .jsdelivr.net
   .microsoft.com
   .sinaapp.com
   .upaiyun.com
   .yandex.st
   .yastatic.net

Put it in `user.action` file.

Instead of using `?!`, the above exception could be equally rewritten like this:

Code:

{ -redirect }
   .googleapis.com/.+/1\.7\.1/

Useful for some websites requiring a specific resource version.

If you encounter an issue while visiting a website, the following alias can help you isolate it:

Code:

JSRoute='egrep "Redirect:\s+pcrs.*changed" /private/var/log/privoxy/logfile.log | \
         sed -r "s/[a-z0-9]+\s+Redirect:[^\"]+\"[^\"]+\"\s+changed\s+/\n/g; s/\s(to)\s/ changed \1…\n/g"| tail -45'

Example of outputs:

Code:

2016-09-05 11:12:07.640
"http://cdnjs.cloudflare.com/ajax/libs/angular.js/1.3.15/angular.min.js" changed to…
"http://localhost/~Your_Path_to/CommonJS/angular.js" (1 hit).
2016-09-05 11:12:07.923
"http://code.jquery.com/ui/1.11.2/jquery-ui.min.js" changed to…
"http://localhost/~Your_Path_to/CommonJS/jquery-ui.js" (1 hit).

Note: do update your libraries regularly…

Some stats.
After one week of surfing…

Code:

Libraries | Request # | JS Sizes
__________|___________|_________
jquery    |    306    x 259.3 KB =  79,345.8 KB
jquery-ui |    156    x 508.5 KB =  79,326   KB
prototype |      7    x 195.1 KB =   1,365.7 KB
angular   |    117    x 144.6 KB =  16,918.2 KB

Total          586               = 176,955.7 KB ≈ 173 MB

173 MB's worth of third-party resources that didn't get downloaded over just one week.

-–—

Minuscule donations are always appreciated…

Code:

BTC --> 34WKogWorDoReJ2MSxw8rTsrGD87VMAPJY
BCH --> 1AXwyMdtMFZktZPvXScC58ESUZXptmjvge
DASH -> XusJsETR6PwDnG4Gde7cvGeRhXzUJFSxtD
ETH --> 0xb829FA99AA9AB31C32590dbc88B837bC5D91453e
ETC --> 0x059F128357331c346Ad2E23F95a4639beC3f0b3a
LTC --> MK7vxk93A1M6HHAYT38W8NPJSb8zANqCia
ZEC --> t1JNCuxdZEWUPBQiAzxZPUMqb4BM87sxs9H
DOGE -> DBPAUuCaez4JYGobAn4RHNNhFXwa9u1W6N
STRAT > SgG6jAHuxQfzW1QBaWyQRVdCdSq514BcyM

[cattleyavns]

Thank you for this very useful tutorial, just one thing I would like to share is, config.privoxy.org/user-manual/ can be used to serve local content, support HTTPS if you open ProxHTTPSProxy's config.ini, find the line with "[SSL No-Verify]" and add:

Code:

config.privoxy.org

And you can access https://config.privoxy.org.
Privoxy have a hidden feature is "user-manual" folder can be used as webserver.

An example:

Code:

FILTER:gmfunc
    s†(^[^;]*?(?:<head[^>]*?>|<body[^>]*?>|<script[^>]*?>[^>]*?<\/script>))†$1\n\
    <script src="http://config.privoxy.org/user-manual/GM_function.js"></script>\n\
    †i

{+filter{gmfunc}}
/

[Faxopita]

(Jan. 05, 2017 08:30 AM)cattleyavns Wrote:  Privoxy have a hidden feature is "user-manual" folder can be used as webserver.

Very good to know indeed! In my case, it's much simpler to use your tip than carrying out the many steps to enable MacOS' local web server…

Thanks.

[kik0s]

(Jan. 05, 2017 08:30 AM)cattleyavns Wrote:  Thank you for this very useful tutorial, just one thing I would like to share is, config.privoxy.org/user-manual/ can be used to serve local content, support HTTPS if you open ProxHTTPSProxy's config.ini, find the line with "[SSL No-Verify]" and add:

Code:

config.privoxy.org

And you can access https://config.privoxy.org.
Privoxy have a hidden feature is "user-manual" folder can be used as webserver.

An example:

Code:

FILTER:gmfunc
    s†(^[^;]*?(?:<head[^>]*?>|<body[^>]*?>|<script[^>]*?>[^>]*?<\/script>))†$1\n\
    <script src="http://config.privoxy.org/user-manual/GM_function.js"></script>\n\
    †i

{+filter{gmfunc}}
/

so i can put the GM file their instead of using greasyfork? but why are you saying it here? does your gm thing does the same what faxtopia showed in the first post?

now i got what you meant with the webserver stuff.

@faxtopia so i can use that commonJS file for every site?! why is it always the same?

[cattleyavns]

(Jan. 14, 2017 11:48 PM)kik0s Wrote:  so i can put the GM file their instead of using greasyfork? but why are you saying it here? does your gm thing does the same what faxtopia showed in the first post?

now i got what you meant with the webserver stuff.

@faxtopia so i can use that commonJS file for every site?! why is it always the same?

I think that time I put greasefork.org link for my GM_function to make sure that the script "can" run on HTTPS site ( the greasefork one is HTTPS ), but a few months from that time, I found out that I can use config.privoxy.org as not only a webserver, but it can work as a HTTPS webserver.

Normally if I use http://config.privoxy.org/user-manual/GM_function.js and try to open Youtube.com, web browser will block GM_function, the reason is, Youtube is HTTPS, the js file is HTTP, HTTPS will block all HTTP (see Mixed Content)

All thank to ProxHTTPSProxy, the above problem is solved.

And I also want to share another filter, this filter will disable Content-Security-Policy on all website, will prevent websites from blocking our local content.

Code:

SERVER-HEADER-FILTER: removecontentsecuritypolicy

[email protected]*@@i

The better way is we can parse Content-Security-Policy and add our local webserver's domain to it, but it is hard for me, but I think someone can do that.

[Faxopita]

(Jan. 14, 2017 11:48 PM)kik0s Wrote:  @faxtopia so i can use that commonJS file for every site?! why is it always the same?

CommonJS is a folder that contains most common used libraries. Privoxy will query the corresponding local versions—inside folder CommonJS—to the remote ones. Web pages will load faster.


If you are not inclined to enable your local web server…

You can copy the content of CommonJS folder to…

Code:

/usr/local/share/doc/privoxy/user-manual/

-–—
I tried copying the folder CommonJS itself, but Privoxy does not, for example, allow access to…

Code:

https://config.privoxy.org/user-manual/CommonJS/angular.js

It doesn't like the folder CommonJS in-between…
—–-

Then, you can replace…

Code:

http://localhost/~Your_Path_to/CommonJS/$2.js

by…

Code:

https://config.privoxy.org/user-manual/$2.js

in the redirect action.

As an example, you can access angular.js by typing

Code:

https://config.privoxy.org/user-manual/angular.js

Notes:

1. I suggest you update the libraries once in a while… A bit tedious, but with a small script it is feasible…
2. If you upgrade Privoxy, be sure to backup your changes, since folder user-manual will likely be overwritten.
   

[kik0s]

@faxtopia looks like the internal webserver is only using one folder. i will check the source if their is a limitation of somekind. so putting the files directly into user-manual should wirk?

[Faxopita]

(Jan. 16, 2017 02:58 PM)kik0s Wrote:  @faxtopia looks like the internal webserver is only using one folder. i will check the source if their is a limitation of somekind. so putting the files directly into user-manual should wirk?

Just tested. I confirm it works.

Make sure you replace

Code:

http://localhost/~Your_Path_to/CommonJS/$2.js

by

Code:

https://config.privoxy.org/user-manual/$2.js

in { +redirect{…} }

Example with angular:

Code:

2017-01-16 15:20:55.539
"http://cdnjs.cloudflare.com/ajax/libs/angular.js/1.3.15/angular.min.js" changed to…
"https://config.privoxy.org/user-manual/angular.js" (1 hit).

[vlad_s]

Files that are redirected, such as specially made?

[vlad_s]

Another way to cancel a redirect. Makes a filter:

Code:

CLIENT-HEADER-TAGGER: do-not-redirect
[email protected]^Referer:\s*.*(site1|site2|site3).*$@[email protected]

where site1|site2|site3 are the sites from which the request was made and you need to cancel the redirect.
Next action:
Activating the filter on yastatic.net and googleapis.com

Code:

{+client-header-tagger{do-not-redirect} \
}
.yastatic.net/
.googleapis.com/

Canceling the redirect when the "test" tag appears:

Code:

{-redirect \
}
TAG:test

Thanks Arcady N. Shpak https://sites.google.com/site/rpfteam/

[ryszardzonk]

(Jan. 15, 2017 03:00 PM)cattleyavns Wrote:  And I also want to share another filter, this filter will disable Content-Security-Policy on all website, will prevent websites from blocking our local content.

Code:

SERVER-HEADER-FILTER: removecontentsecuritypolicy

[email protected]*@@i

The better way is we can parse Content-Security-Policy and add our local webserver's domain to it, but it is hard for me, but I think someone can do that.

I like that, but it seems not to work as websites still give messages of disabled content

mixedcontent.filter

Code:

SERVER-HEADER-FILTER: removecontentsecuritypolicy
[email protected]*@@i

mixedcontent.action

Code:

{+server-header-filter{removecontentsecuritypolicy}}
192.168.1.1

I server with privoxy same content to the webpage with ssl for devices that accept local certificates and without ssl for those that don't so at least http only content would keep filtered there, but getting this to work would mean I could use only http for both.
Would I need to disable it for webpages I visit? I would like to avoid that as disabling seurity policies on web pages seems not as the best idea.

On other side have You been able to write the code that would inject information about local server as trusted into the webpage code?

[ryszardzonk]

@Faxopita & @cattleyavns

(Jan. 01, 2017 12:56 PM)Faxopita Wrote:  To roughly mimic “decentraleyes” Firefox add-on…

Seems that BetterCache approach proposed by cattleyavns could also achieve the same and hence if I understand it right cached files are there until next privoxy restart than by restarting once a month one could have both freshest libraries which would be downloaded only once by privoxy when needed.

[cattleyavns]

(Jul. 16, 2018 09:03 PM)ryszardzonk Wrote:  I like that, but it seems not to work as websites still give messages of disabled content

I just want to correct about "prevent Content-Security-Policy from blocking local contents", this feature prevent "Content-Security-Policy's rules" (see https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP ) from blocking our local contents and not "Mixed Content", the only way to bypass "Mixed Content" on HTTPS websites is to make our local contents become HTTPS.

[ryszardzonk]

I actually tried it for static fonts as they do not change, but seems it does not work. What am I doing wrong?

Code:

2018-08-14 21:47:12.444 7fc5e27fc700 Request: encrypted-tbn3.gstatic.com/shopping?q=tbn:ANd9GcRsGohMOn8Ns7Wivy1PWEDrP2NrCHiu-pFfzY0x4l9LyEB5fJVvjrxHgE77AOXH3DHWVGAW4hs&usqp=CAc
2018-08-14 21:47:24.916 7fc61d50c700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxK.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:24.916 7fc61d50c700 Request: fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxK.woff2
2018-08-14 21:47:34.615 7fc61e50e700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOUuhp.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:34.615 7fc61e50e700 Request: fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOUuhp.woff2
2018-08-14 21:47:35.186 7fc61ed0f700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOXOhpOqc.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:35.186 7fc61ed0f700 Request: fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOXOhpOqc.woff2

decentraleyes.action

Code:

#P5sMzZCDf9_T_10XxCF8jA.woff2
#P5sMzZCDf9_T_10ZxCE.woff2
#P5sBzZCDf9_T_1Wi4TRDrZI.woff2
#P5sBzZCDf9_T_1Wi4TRNrZKX0A.woff2
#S6uyw4BMUTPHjx4wXg.woff2
#S6uyw4BMUTPHjxAwXjeu.woff2
#TK3hWkUHHAIjg75-ohoTus9C.woff2
{ +redirect{[email protected]^(http?://)?[^,%]+\b(P5sMzZCDf9_T_1(0XxCF8jA|0ZxCE|Wi4TRDrZI|Wi4TRNrZKX0A)|S6uyw4BMUTPHjx(4wXg|AwXjeu)|TK3hWkUHHAIjg75-ohoTus9C)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]} }
fonts\.gstatic\.com

#u-440qyriQwlOrhSvowK_l5-ciZMZ-Y.woff2
#u-440qyriQwlOrhSvowK_l5-fCZM.woff2
#u-4m0qyriQwlOrhSvowK_l5-eRZOf-I.woff2
#u-4n0qyriQwlOrhSvowK_l52_wFZVsf6lvg.woff2
#u-4n0qyriQwlOrhSvowK_l52_wFZWMf6.woff2
#u-4n0qyriQwlOrhSvowK_l52xwNZVsf6lvg.woff2
#u-4n0qyriQwlOrhSvowK_l52xwNZWMf6.woff2
{ +redirect{[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]} }
fonts\.gstatic\.com

[cattleyavns]

(Aug. 14, 2018 07:54 PM)ryszardzonk Wrote:  I actually tried it for static fonts as they do not change, but seems it does not work. What am I doing wrong?

Code:

2018-08-14 21:47:12.444 7fc5e27fc700 Request: encrypted-tbn3.gstatic.com/shopping?q=tbn:ANd9GcRsGohMOn8Ns7Wivy1PWEDrP2NrCHiu-pFfzY0x4l9LyEB5fJVvjrxHgE77AOXH3DHWVGAW4hs&usqp=CAc
2018-08-14 21:47:24.916 7fc61d50c700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxK.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:24.916 7fc61d50c700 Request: fonts.gstatic.com/s/roboto/v18/KFOmCnqEu92Fr1Mu4mxK.woff2
2018-08-14 21:47:34.615 7fc61e50e700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOUuhp.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:34.615 7fc61e50e700 Request: fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOUuhp.woff2
2018-08-14 21:47:35.186 7fc61ed0f700 Error: pcrs command "[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]" changed "http://fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOXOhpOqc.woff2" to "[null]" (179 hits), but the result doesn't look like a valid URL and will be ignored.
2018-08-14 21:47:35.186 7fc61ed0f700 Request: fonts.gstatic.com/s/opensans/v15/mem5YaGs126MiZpBA-UNirkOXOhpOqc.woff2

decentraleyes.action

Code:

#P5sMzZCDf9_T_10XxCF8jA.woff2
#P5sMzZCDf9_T_10ZxCE.woff2
#P5sBzZCDf9_T_1Wi4TRDrZI.woff2
#P5sBzZCDf9_T_1Wi4TRNrZKX0A.woff2
#S6uyw4BMUTPHjx4wXg.woff2
#S6uyw4BMUTPHjxAwXjeu.woff2
#TK3hWkUHHAIjg75-ohoTus9C.woff2
{ +redirect{[email protected]^(http?://)?[^,%]+\b(P5sMzZCDf9_T_1(0XxCF8jA|0ZxCE|Wi4TRDrZI|Wi4TRNrZKX0A)|S6uyw4BMUTPHjx(4wXg|AwXjeu)|TK3hWkUHHAIjg75-ohoTus9C)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]} }
fonts\.gstatic\.com

#u-440qyriQwlOrhSvowK_l5-ciZMZ-Y.woff2
#u-440qyriQwlOrhSvowK_l5-fCZM.woff2
#u-4m0qyriQwlOrhSvowK_l5-eRZOf-I.woff2
#u-4n0qyriQwlOrhSvowK_l52_wFZVsf6lvg.woff2
#u-4n0qyriQwlOrhSvowK_l52_wFZWMf6.woff2
#u-4n0qyriQwlOrhSvowK_l52xwNZVsf6lvg.woff2
#u-4n0qyriQwlOrhSvowK_l52xwNZWMf6.woff2
{ +redirect{[email protected]^(http?://)?[^,%]+\b(u-440qyriQwlOrhSvowK_l5-(ciZMZ-Y|fCZM)|u-4m0qyriQwlOrhSvowK_l5-eRZOf-I|u-4n0qyriQwlOrhSvowK_l52(_wFZVsf6lvg|_wFZWMf6|xwNZVsf6lvg|xwNZWMf6)\b[^/]*\.woff2[^,%]*@https://192.168.101.101:8447/font/$[email protected]} }
fonts\.gstatic\.com

There is something wrong with your Regex rule, I think modify it again is the way to go.