Post Reply 
ProxHTTPSProxyMII: Reloaded
Dec. 07, 2018, 03:40 PM
Post: #301
RE: ProxHTTPSProxyMII: Reloaded
SOLVED!

I've made some progress, after my "random thought" last night... Using the Dev Tools panel in Firefox, I loaded https://www.google.com. Here are the request headers and the Privoxy results log:

Code:
Host: www.google.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache

2018-12-07 10:12:40.177 00002080 Request: www.google.com/
2018-12-07 10:12:40.323 00002080 Re-Filter: Need to de-chunk first
2018-12-07 10:12:40.330 00002080 Re-Filter: De-chunking successful. Shrunk from 63991 to 63978
2018-12-07 10:12:40.337 00002080 Re-Filter: filtering www.google.com/ (size 63978) with 'removeanchortargets' produced 0 hits (new size 63978).
2018-12-07 10:12:40.343 00002080 Re-Filter: filtering www.google.com/ (size 63978) with 'removeemptyalts' produced 0 hits (new size 63978).
2018-12-07 10:12:40.348 00002080 Re-Filter: filtering www.google.com/ (size 63978) with 'TEST' produced 0 hits (new size 63978).

Note that none of the three filters had any matches.

I then made exactly the same request again, with one alteration to the headers:

Code:
Host: www.google.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
[b]Accept-Encoding: gzip[/b]
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache

2018-12-07 10:13:17.670 00002080 Request: www.google.com/
2018-12-07 10:13:17.815 00002080 Re-Filter: Need to de-chunk first
2018-12-07 10:13:17.823 00002080 Re-Filter: De-chunking successful. Shrunk from 69356 to 69335
2018-12-07 10:13:17.834 00002080 Re-Filter: Decompression successful. Old size: 69335, new size: 219348.
2018-12-07 10:13:17.854 00002080 Re-Filter: filtering www.google.com/ (size 219348) with 'removeanchortargets' produced 1 hits (new size 219333).
2018-12-07 10:13:17.859 00002080 Re-Filter: filtering www.google.com/ (size 219333) with 'removeemptyalts' produced 0 hits (new size 219333).
2018-12-07 10:13:17.866 00002080 Re-Filter: filtering www.google.com/ (size 219333) with 'TEST' produced 230 hits (new size 219793).

Now we see that two of the filters DID have matches, and the filtering worked! What changed? This "Decompression successful. Old size: 69335, new size: 219348" was not done in the first request, so Privoxy was trying to filter compressed content. Either deflate or br. With that in mind, I checked the options for Firefox, to see if it allows configurations of what encodings it supports, and found under about:config this option: "network.http.accept-encoding.secure;gzip, deflate, br". I removed "br" from the end, and now my HTTPS filters work!!

So I guess that Privoxy simply does not support Brotli encoding.
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to Quaraxkad for this post:
referrer
Post Reply 


Forum Jump: