Threaded Mode | Linear Mode

ryszardzonk · Aug. 21, 2018, 12:14 PM

First part is definitely OK as input has been caught. From the error it seems is the replacement part that wrong as it says resulting URL is invalid.
Could it be that when using "$2" variable for both cases is wrong and instead of using @https://192.168.101.101:8447/font/$2.woff2@ for both one should have $1 while other $2?

vlad_s · Mar. 31, 2019, 06:34 PM

Here is this addon https://addons.mozilla.org/ru/firefox/ad...ntraleyes/
I downloaded the xpi file and unpacked it. Decided to do actions and filters. But, with yastatic.net the redirect does not work. In the privoxy log reports that everything is done, but the file itself is not harvested, it can be seen in the apache2 log. Here is the action:

Code:

{+redirect{s@^(https?://)?[^,%]+\b(angularjs|jquery|modernizr|moment)(js)?/([0-9\.]+)/[a-z\-]+\b[^/]*\.js$@https://192.168.2.1/decentraleyes/$2/$4/$2.min.jsm@} \

}

yastatic.net/(angularjs|jquery|modernizr|momentjs)/

Through addon HttpFox, I saw the message type "application/x-javascript (NS_ERROR_DOM_BAD_URI)" in the type column.

What could be the problem?

***JJoe*** · Apr. 01, 2019, 05:39 AM

(Mar. 31, 2019 06:34 PM)vlad_s Wrote: What could be the problem?

Have you added 192.168.2.1 to yandex's Content-Security-Policy header as an allowed script-src?

Yandex's header:

Code:

Content-Security-Policy: connect-src 'self' wss://webasr.yandex.net https://mc.webvisor.com https://mc.webvisor.org wss://push.yandex.ru wss://portal-xiva.yandex.net https://yastatic.net 

https://home.yastatic.net https://yandex.ru https://*.yandex.ru static.yandex.sx brotli.yastatic.net et.yastatic.net *.serving-sys.com an.yandex.ru awaps.yandex.ru storage.mds.yandex.net music.yandex.ru music-browser.music.yandex.net mc.admetrica.ru portal-xiva.yandex.net yastatic.net home.yastatic.net yandex.ru *.yandex.ru *.yandex.net yandex.st; default-src 'self' blob: wss://portal-xiva.yandex.net yastatic.net portal-xiva.yandex.net; font-src 'self' https://yastatic.net zen.yandex.ru static.yandex.sx brotli.yastatic.net et.yastatic.net yabro1.zen-test.yandex.ru main.zdevx.yandex.ru yastatic.net; frame-src 'self' yabrowser: data: https://ok.ru https://www.youtube.com https://player.video.yandex.net https://ya.ru https://yastatic.net https://yandex.ru https://*.yandex.ru https://downloader.yandex.net wfarm.yandex.net secure-ds.serving-sys.com yandexadexchange.net *.yandexadexchange.net music.yandex.ru yastatic.net yandex.ru *.yandex.ru awaps.yandex.net *.cdn.yandex.net; img-src 'self' data: https://yastatic.net https://home.yastatic.net https://*.yandex.ru https://*.yandex.net https://*.tns-counter.ru awaps.yandex.net *.yastatic.net gdeua.hit.gemius.pl pa.tns-ua.com mc.yandex.com mc.webvisor.com mc.webvisor.org static.yandex.sx brotli.yastatic.net et.yastatic.net *.moatads.com avatars.mds.yandex.net bs.serving-sys.com an.yandex.ru awaps.yandex.ru nissanhelioseurope.demdex.net mc.admetrica.ru yastatic.net home.yastatic.net yandex.ru *.yandex.ru *.yandex.net *.tns-counter.ru yandex.st; media-src 'self' blob: data: *.storage.yandex.net *.yandex.net strm.yandex.ru strm.yandex.net *.strm.yandex.net *.cdn.yandex.net storage.mds.yandex.net *.storage.mds.yandex.net yastatic.net kiks.yandex.ru; object-src 'self' *.yandex.net music.yandex.ru strm.yandex.ru flashservice.adobe.com yastatic.net kiks.yandex.ru awaps.yandex.net storage.mds.yandex.net; report-uri https://csp.yandex.net/csp?from=big.ru&showid=1554095301.32498.140931.438799&h=vla1-8317-5da-vla-portal-morda-31387&csp=old&date=20190401&yandexuid=4458360781554095089; 

script-src 'self' 'unsafe-inline' 'unsafe-eval' 

blob: https://suburban-widget.rasp.yandex.ru https://suburban-widget.rasp.yandex.net https://music.yandex.ru https://mc.yandex.fr https://mc.webvisor.com https://yandex.fr https://mc.webvisor.org https://yastatic.net https://home.yastatic.net https://mc.yandex.ru https://pass.yandex.ru zen.yandex.ru an.yandex.ru api-maps.yandex.ru static.yandex.sx webasr.yandex.net brotli.yastatic.net et.yastatic.net z.moatads.com bs.serving-sys.com secure-ds.serving-sys.com yabro1.zen-test.yandex.ru main.zdevx.yandex.ru awaps.yandex.ru storage.mds.yandex.net yastatic.net home.yastatic.net yandex.ru www.yandex.ru mc.yandex.ru suggest.yandex.ru clck.yandex.ru awaps.yandex.net; style-src 'self' 'unsafe-inline' https://yastatic.net https://home.yastatic.net zen.yandex.ru static.yandex.sx brotli.yastatic.net et.yastatic.net yabro1.zen-test.yandex.ru main.zdevx.yandex.ru yastatic.net home.yastatic.net;

vlad_s · (This post was last modified: Apr. 01, 2019 04:12 PM by vlad_s.)

Yes added:

Code:

SERVER-HEADER-FILTER: Content-Security-Policy Content Security Policy.

s@^(Content-Security-Policy:\s+.*)$@$1 192.168.2.1@i

In the Privoxy log writes:

Code:

Header: Transforming "Content-Security-Policy: default-src 'none'; script-src 'self'...mc... [too long, truncated]

And I tried to delete all these extra headers, I did not achieve the result. Maybe something is not right?

vlad_s · (This post was last modified: Apr. 01, 2019 04:44 PM by vlad_s.)

In general, the header is modified, but still does not work.

From Iron browser:

Code:

Refused to load the script 'https://192.168.2.1/decentraleyes/jquery/2.2.3/jquery.min.jsm' because it violates the following Content Security Policy directive: "script-src 'self' 'unsafe-inline' 'unsafe-eval' 'nonce-6JP75lFRjo5x8uG1ExrL7g' yastatic.net *.yandex.net yandex.st *.yandex.st yandex.ru *.yandex.ru mc.yandex.ru mc.yandex.by mc.yandex.kz mc.yandex.ua mc.yandex.co.il mc.yandex.com mc.yandex.com.am mc.yandex.com.ge mc.yandex.com.tr mc.yandex.az mc.yandex.ee mc.yandex.fr mc.yandex.kg mc.yandex.lt mc.yandex.lv mc.yandex.md mc.yandex.ru mc.yandex.tj mc.yandex.tm mc.yandex.uz mc.webvisor.com mc.webvisor.org yastat.net pass.yandex.ru suggest.yandex.ru news.yandex.ru social.yandex.ru export.yandex.ru an.yandex.ru awaps.yandex.ru yabs.yandex.ru news-clck.yandex.ru".

***JJoe*** · Apr. 02, 2019, 04:15 AM

(Apr. 01, 2019 04:11 PM)vlad_s Wrote: And I tried to delete all these extra headers, I did not achieve the result. Maybe something is not right?

If the browser blocks 192.168.2.1 by default,
deleting the header may not help.

(Apr. 01, 2019 04:35 PM)vlad_s Wrote: In general, the header is modified,

Are you sure? This message from Iron does not show 192.168.2.1 in script-src section of the Content Security Policy directive.

(Apr. 01, 2019 04:35 PM)vlad_s Wrote:

Code:

Refused to load the script 'https://192.168.2.1/decentraleyes/jquery/2.2.3/jquery.min.jsm' because it violates the following Content Security Policy directive: "script-src 'self' 'unsafe-inline' 'unsafe-eval' 'nonce-6JP75lFRjo5x8uG1ExrL7g' yastatic.net *.yandex.net yandex.st *.yandex.st yandex.ru *.yandex.ru mc.yandex.ru mc.yandex.by mc.yandex.kz mc.yandex.ua mc.yandex.co.il mc.yandex.com mc.yandex.com.am mc.yandex.com.ge mc.yandex.com.tr mc.yandex.az mc.yandex.ee mc.yandex.fr mc.yandex.kg mc.yandex.lt mc.yandex.lv mc.yandex.md mc.yandex.ru mc.yandex.tj mc.yandex.tm mc.yandex.uz mc.webvisor.com mc.webvisor.org yastat.net pass.yandex.ru suggest.yandex.ru news.yandex.ru social.yandex.ru export.yandex.ru an.yandex.ru awaps.yandex.ru yabs.yandex.ru news-clck.yandex.ru".

vlad_s · (This post was last modified: Apr. 02, 2019 07:01 PM by vlad_s.)

The exact sequence of my actions. In the file filter:

Code:

SERVER-HEADER-FILTER: Content-Security-Policy Content Security Policy.

s@^(Content-Security-Policy:\s+.*)$@$1 192.168.2.1@i

To comply with the "Content Security Policy directive". There are two actions in the file action:

Code:

{+server-header-filter{Content-Security-Policy} \

}

.yandex.*/

.yastatic.*/

{+redirect{s@^(https?://)?[^,%]+\b(angularjs|jquery|modernizr|moment)(js)?/([0-9\.]+)/[a-z\-]+\b[^/]*\.js$@https://192.168.2.1/decentraleyes/$2/$4/$2.min.jsm@} \

}

yastatic.net/(angularjs|jquery|modernizr|momentjs)/

I get this:

It can be seen that my added ip 192.168.2.1 appears somewhere, but the script does not load.
And I can see in the Apache log that the request for https://192.168.2.1/decentraleyes/jquery...ry.min.jsm does not even pass, but a message in the console like "Refused to load the script" https: // 192.168.2.1/decentraleyes/jquery/2.1.4/jquery.min.jsm ..." appears. How does the browser know that something is wrong with 192.168.1, because it did not download the script from this link to find out the Content Security Policy header?

I added another filter to eliminate the message "Refused to load the script" https: // 192.168.2.1/decentraleyes/jquery/2.1.4/jquery.min.jsm ... ":

Code:

SERVER-HEADER-FILTER: Add-Content-Security-Policy Add header Content Security Policy.

s@^(Content-Length:\s+.*)$@$1\r\nContent-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' 'nonce-*' yastatic.net

*.yandex.net yandex.st *.yandex.st yandex.ru *.yandex.ru mc.yandex.ru mc.yandex.by mc.yandex.kz mc.yandex.ua mc.yandex.co.il mc

.yandex.com mc.yandex.com.am mc.yandex.com.ge mc.yandex.com.tr mc.yandex.az mc.yandex.ee mc.yandex.fr mc.yandex.kg mc.yandex.lt

mc.yandex.lv mc.yandex.md mc.yandex.ru mc.yandex.tj mc.yandex.tm mc.yandex.uz mc.webvisor.com mc.webvisor.org yastat.net pass.ya

ndex.ru suggest.yandex.ru news.yandex.ru social.yandex.ru export.yandex.ru an.yandex.ru awaps.yandex.ru yabs.yandex.ru news-clck

.yandex.ru 192.168.2.1@i

and action:

Code:

{+server-header-filter{Content-Security-Policy} \

}

.yandex.*/

.yastatic.*/

{+server-header-filter{Add-Content-Security-Policy} \

}

192.168.2.1/decentraleyes/

I get this:

I load the script using the link https://192.168.2.1/decentraleyes/jquery...ry.min.jsm to see the headers:

It seems all right? But does not work.

***JJoe*** · Apr. 03, 2019, 12:40 AM

Ahh... I don't think you understand.
You need to modify the Content Security Policy response header from yandex.
You do not need to add the header to 192.168.2.1

(Apr. 02, 2019 05:59 PM)vlad_s Wrote: It can be seen that my added ip 192.168.2.1 appears somewhere, but the script does not load.

192.168.2.1 needs to be in the "script-src" section of the response header from the yandex server.

(Apr. 02, 2019 05:59 PM)vlad_s Wrote: How does the browser know that something is wrong with 192.168.1, because it did not download the script from this link to find out the Content Security Policy header?

The Content Security Policy header belongs to the response from yandex.ru.
It tells the browser the addresses that it may access to create the page.

The script is not allowed because the address 192.168.2.1. is not allowed.

An example from the Proxomitron:
The sidki set uses files from https://local.ptron:8443 .
So, it adds "https://local.ptron:8443" for 'default-src', 'img-src', 'script-src', and 'style-src'.
This is the header that the set creates from the 'yandex.ru' response header:

Quote:Content-Security-Policy: connect-src 'self' wss://webasr.yandex.net
https://mc.webvisor.com https://mc.webvisor.org wss://push.yandex.ru wss://portal-xiva.yandex.net
https://yastatic.net https://home.yastatic.net https://yandex.ru https://*.yandex.ru
static.yandex.sx brotli.yastatic.net et.yastatic.net *.serving-sys.com an.yandex.ru
awaps.yandex.ru storage.mds.yandex.net music.yandex.ru music-browser.music.yandex.net
mc.admetrica.ru portal-xiva.yandex.net yastatic.net home.yastatic.net yandex.ru *.yandex.ru
*.yandex.net yandex.st; default-src https://local.ptron:8443 'unsafe-inline' 'self'
blob: wss://portal-xiva.yandex.net yastatic.net portal-xiva.yandex.net; font-src
'self' https://yastatic.net zen.yandex.ru static.yandex.sx brotli.yastatic.net et.yastatic.net
yabro1.zen-test.yandex.ru main.zdevx.yandex.ru yastatic.net; frame-src 'self' yabrowser:
data: https://ok.ru https://www.youtube.com https://player.video.yandex.net https://ya.ru
https://yastatic.net https://yandex.ru https://*.yandex.ru https://downloader.yandex.net
wfarm.yandex.net secure-ds.serving-sys.com yandexadexchange.net *.yandexadexchange.net
music.yandex.ru yastatic.net yandex.ru *.yandex.ru awaps.yandex.net *.cdn.yandex.net;
img-src https://local.ptron:8443 'unsafe-inline' 'self' data: https://yastatic.net
https://home.yastatic.net https://*.yandex.ru https://*.yandex.net https://*.tns-counter.ru
awaps.yandex.net *.yastatic.net gdeua.hit.gemius.pl pa.tns-ua.com mc.yandex.com mc.webvisor.com
mc.webvisor.org static.yandex.sx brotli.yastatic.net et.yastatic.net *.moatads.com
avatars.mds.yandex.net bs.serving-sys.com an.yandex.ru awaps.yandex.ru nissanhelioseurope.demdex.net
mc.admetrica.ru yastatic.net home.yastatic.net yandex.ru *.yandex.ru *.yandex.net
*.tns-counter.ru yandex.st; media-src 'self' blob: data: *.storage.yandex.net *.yandex.net
strm.yandex.ru strm.yandex.net *.strm.yandex.net *.cdn.yandex.net storage.mds.yandex.net
*.storage.mds.yandex.net yastatic.net kiks.yandex.ru; object-src 'self' *.yandex.net
music.yandex.ru strm.yandex.ru flashservice.adobe.com yastatic.net kiks.yandex.ru
awaps.yandex.net storage.mds.yandex.net; script-src https://local.ptron:8443 'unsafe-inline'
'self' 'unsafe-inline' 'unsafe-eval' blob: https://suburban-widget.rasp.yandex.ru
https://suburban-widget.rasp.yandex.net https://music.yandex.ru https://mc.yandex.fr
https://mc.webvisor.com https://yandex.fr https://mc.webvisor.org https://yastatic.net
https://home.yastatic.net https://mc.yandex.ru https://pass.yandex.ru zen.yandex.ru
an.yandex.ru api-maps.yandex.ru static.yandex.sx webasr.yandex.net brotli.yastatic.net
et.yastatic.net z.moatads.com bs.serving-sys.com secure-ds.serving-sys.com yabro1.zen-test.yandex.ru
main.zdevx.yandex.ru awaps.yandex.ru storage.mds.yandex.net yastatic.net home.yastatic.net
yandex.ru www.yandex.ru mc.yandex.ru suggest.yandex.ru clck.yandex.ru awaps.yandex.net;
style-src https://local.ptron:8443 'unsafe-inline' 'self' 'unsafe-inline' https://yastatic.net
https://home.yastatic.net zen.yandex.ru static.yandex.sx brotli.yastatic.net et.yastatic.net
yabro1.zen-test.yandex.ru main.zdevx.yandex.ru yastatic.net home.yastatic.net;

To do this the filter replaces
'default-src' with 'default-src https://local.ptron:8443 ',
'img-src' with 'img-src https://local.ptron:8443 ',
'script-src' with 'script-src https://local.ptron:8443 ',
'style-src' with 'style-src https://local.ptron:8443 '

It also adds 'unsafe-inline'.

vlad_s · Apr. 03, 2019, 05:10 PM

You are right, I am not well versed in this. But I did as you indicated. The script "https://yastatic.net/jquery/2.1.4/jquery.min.js" does not load due to the absence of another header, "Access-Control-Allow-Origin: *". I added it:

Code:

#filter

SERVER-HEADER-FILTER: Add-Access-Control-Allow-Origin Add Access-Control-Allow-Origin header

s@^(Content-Length:\s+.*)$@$1\r\nAccess-Control-Allow-Origin: *@i

#action

{-filter{banners-by-size} \

+server-header-filter{Add-Content-Security-Policy} \

}

.yandex.*/

{+server-header-filter{Add-Access-Control-Allow-Origin} \

}

yastatic.net/jquery/

But the error did not change. Maybe I'm doing something wrong again?

***JJoe*** · (This post was last modified: Apr. 04, 2019 02:18 AM by JJoe.)

(Apr. 03, 2019 05:10 PM)vlad_s Wrote: I am not well versed in this.

"This" is always changing. So, we are all always learning. Smile!

I'm not sure Privoxy can do this and as I understand it... Wink

Your browser requests yastatic.net/jquery/2.1.4/jquery.min.js.
Privoxy returns a redirect (302) to https://192.168.2.1/decentraleyes/jquery/2.1.4/jquery.min.js.
Your browser requests https://192.168.2.1/decentraleyes/jquery/2.1.4/jquery.min.js.
192.168.2.1 returns (200) jquery.min.js.

Privoxy's redirect response (302) AND 192.168.2.1 server's file response (200)
must contain

Code:

Access-Control-Allow-Origin: *

Can Access-Control-Allow-Origin header be added to Privoxy's redirect (302) response?

BTW:
The sidki set adds 'unsafe-inline' for the inline scripts that it adds to the page. The next set will not add 'unsafe-inline'.

If you don't need 'unsafe-inline', don't add it.

vlad_s · (This post was last modified: Apr. 04, 2019 06:49 PM by vlad_s.)

No, when requested by the browser yastatic.net/jquery/2.1.4/jquery.min.js, it does not actually occur. In the Apache log, there is no record of this, and when everything is correct, then the logs contain something like this:

Code:

192.168.2.1 - - [03/Apr/2019:21:41:21 +0300] "GET /decentraleyes/jquery/1.11.3/jquery.min.jsm HTTP/1.1" 200 97729 "http://ipv6-test.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0"

That is, the browser does not even try to download files from a new address. Privoxy - I could not add a header to 302, in at least the same way as it allows (via SERVER-HEADER-FILTER). Although I definitely did not check it. I don't know what 'unsafe-inline' is doing and I have Linux Smile!

. Thanks for the support and patience. I will continue...

***JJoe*** · (This post was last modified: Apr. 04, 2019 11:27 PM by JJoe.)

(Apr. 04, 2019 06:48 PM)vlad_s Wrote: Privoxy - I could not add a header to 302,

The missing header in Privoxy's 302 response is the problem.

If a browser extension is acceptable,
CORS Everywhere for Firefox should allow the file to load.

Try adding https://192.168.2.1 or https://192.168.2.1:443 to the extension's Activation whitelist.

vlad_s · Apr. 05, 2019, 07:23 PM

Is it possible to somehow weaken the site policy by removing some of the headers?

***JJoe*** · (This post was last modified: Apr. 06, 2019 03:55 AM by JJoe.)

(Apr. 05, 2019 07:23 PM)vlad_s Wrote: Is it possible to somehow weaken the site policy by removing some of the headers?

I don't think that will work. We can remove the request headers but the browser still knows that it sent them.

However, redirect works after I change Yandex.ru html from

Code:

<link rel="preload" href="//yastatic.net/jquery/2.1.4/jquery.min.js" as="script" crossorigin="anonymous">

...

<script src="//yastatic.net/jquery/2.1.4/jquery.min.js" crossorigin="anonymous">

to

Code:

<link rel="preload" href="//yastatic.net/jquery/2.1.4/jquery.min.js" as="script">

...

<script src="//yastatic.net/jquery/2.1.4/jquery.min.js">

vlad_s · (This post was last modified: Apr. 08, 2019 06:41 PM by vlad_s.)

It works. I have a bad understanding of this, but I looked at news.yandex.ru, there is code "301 Moved Permanently" for the request https://news.yandex.ru/favicon.ico for https://yastatic.net/iconostasis/_/egVpf...c22BBg.ico without transmitting the header "Access-Control-Allow-Origin: *". I am assuming that you can still configure the redirect, without filtering the page suggested with your last message? Or am I wrong?