ProxHTTPSProxyMII: Development

[JJoe]

(Jun. 17, 2014 10:06 AM)whenever Wrote:  Let's be a little more polite by changing line 119 of proxytool.py from

I just saw and logged it, after changing line 119. Maybe some context will help.

Some of these problems seem to happen only while web page filtering is enabled.

For testing, I have added another instance of the Proxomitron.
Looks like FrontProxHTTPSProxy>firstProxomitron>secondProxomitron>RearProxHTTPSProxy.
firstProxomitron only logs, no user filters. secondProxomitron may filter.

A request for the yahoo mail page after signing in looks like:

Code:

********firstProxomitron*********
+++GET 3+++
Using Proxy - 127.0.0.1:8080
GET http://us-mg.mail.yahoo.com/neo/b/launch?.rand=9 HTTP/1.1
Accept-Encoding: gzip,deflate,lzma,sdch
Host: us-mg.mail.yahoo.com
Referer: https://login.yahoo.com/?mbsw=d&.src=ym&.intl=us
Cookie:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.8
Tagged: Proxomitron FrontProxy/0.8d Python/3.4.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.132 Safari/537.36 OPR/21.0.1432.57 (Edition Campaign 37)
Connection: keep-alive
********/firstProxomitron********

********secondProxomitron********
+++GET 3+++
Using Proxy - 127.0.0.1:8081
GET http://us-mg.mail.yahoo.com/neo/b/launch?.rand=9 HTTP/1.1
Accept-Encoding: gzip, deflate
Host: us-mg.mail.yahoo.com
Referer: https://login.yahoo.com/?mbsw=d&.src=ym&.intl=us
Cookie:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.8
Tagged: Proxomitron FrontProxy/0.8d Python/3.4.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.132 Safari/537.36 OPR/21.0.1432.57 (Edition Campaign 37)
Connection: keep-alive
** 3 Socket Error 10053 for request flush **

+++RESP 3+++
HTTP/1.1 200 OK
Date: Tue, 17 Jun 2014 21:21:14 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Expires: Wed, 19 Apr 1970 17:30:00 GMT
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Age: 1
Server: ATS
Via: http/1.1 r13.ycpi.ne1.yahoo.net (ApacheTrafficServer/4.0.2 [cMsSf ])
Content-Length: 13864
Connection: Keep-Alive
Match 3: New HTML filter
********/secondProxomitron********

********firstProxomitron********
+++RESP 3+++
HTTP/1.1 200 OK
Date: Tue, 17 Jun 2014 21:21:14 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Expires: Wed, 19 Apr 1970 17:30:00 GMT
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Age: 1
Server: ATS
Via: http/1.1 r13.ycpi.ne1.yahoo.net (ApacheTrafficServer/4.0.2 [cMsSf ])
Transfer-Encoding: chunked
********/firstProxomitron********

Summary:
secondProxomitron user header filter changes "Accept-Encoding" header from "gzip,deflate,lzma,sdch" to "gzip, deflate"
** 3 Socket Error 10053 for request flush **, this does not always happen.
"Content-Encoding", "Content-Length" and Connection headers are removed but not by user filters. This is what I remember and expected.
"Transfer-Encoding: chunked" header is added, as expected.

Not shown:
When the secondProxomiton is not filtering, only the Connection header is missing in firstProxomitron log window.

So could the switch to "Transfer-Encoding: chunked" be a problem?

I'll be looking at \k next.