ProxHTTPSProxy, a Proxomitron SSL Helper Program

[Gravemind]

Quote:- If you want to filter all https, set your browser's https proxy to ProxHTTPSProxy
Code:
# ProxHTTPSProxy
[^/]++:443&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

Anybody alive here?

Could you please export your Proxomitron rules for this ProxHTTPSProxy and post it in plain text.

I am trying to use the advised rule in the outgoing header and I only get errors.

People like me can`t think too much. So, a plain rule and a note on how to chain it would be nice.

[ProxRocks]

(May. 26, 2012 01:56 PM)Gravemind Wrote:  Anybody alive here?

absolutely

i haven't used the ProxHTTPSProxy program in quite some time so i'll wait to see if others chime in first...

[JJoe]

(May. 26, 2012 01:56 PM)Gravemind Wrote:  Could you please export your Proxomitron rules for this ProxHTTPSProxy and post it in plain text.

Those instructions assume that you are using sidki's set...

Quote:This is how it look like in sidki's Exceptions-U.ptxt:

- If you want to filter all https, set your browser's https proxy to ProxHTTPSProxy
Code:

http://prxbx.com/forums/showthread.php?tid=1870

Assuming that you are not using sidki's but have installed ProxHTTPSProxy and all it's supporting files...

Add ProxHTTPSProxy to the Proxomitron via the "External Proxy Selector", http://proxomitron.info/45/help/External...ialog.html .

Then try adding a filter like

Code:

[HTTP headers]
In = FALSE
Out = TRUE
Key = "! |||||||||||| Use ProxHTTPSProxy  (fail) (Out)"
URL = "[^/]++:443&$URL(http://*)  $SETPROXY(127.0.0.1:8081) (^)"

http://proxomitron.info/45/help/Matching...l#SETPROXY

HTH

[Gravemind]

I made it work, sort of, but not really.

Like in the manual, I installed the python (even though I had a compiled executable downloaded) and openssl.

The proxy tests and runs fine. It did even without Python installed.

Concerning the rules:

This rule didn`t redirect anything. I tried changing it in different ways, but it was hopeless.

But the redirecting rules are not that important, I figured there were ways.

I have, for example, 2 lame rules, which show some activity in the ProxHTTPSProxy window. Which is some progress already.

The rules are:

Quote:[HTTP headers]
In = FALSE
Out = TRUE
Key = "ProxHTTPSProxy 443 http"
URL = "$URL(http:?++443:?+) $SETPROXY(127.0.0.1:8081)"

Quote:[HTTP headers]
In = FALSE
Out = TRUE
Key = "ProxHTTPSProxy HTTPS"
URL = "$URL(https://?+) $SETPROXY(127.0.0.1:8081)"

The In/Out combinations don`t matter — they don`t work anyway. I am missing something here.

Quote:Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

Hint: https://workflowy.com/

Apache/2.2.12 (Ubuntu) Server at workflowy.com Port 443

But the easy way to make it work is to check "Use Remote Proxy". Then it starts working, the sites are all moved though it.

But it doesn`t issue 307 redirects, I guess.

Because the browser keeps showing https-warning windows. That means, it still believes it is on a secured page. Which is not supposed to happen, right?

If I make an exception for the certificate, it continues working and really changes links into http like this:

https://workflowy.com/
http://workflowy.com:443/

https://click.alfabank.ru/ALFAIBSR/
http://click.alfabank.ru:443/ALFAIBSR/

From the log below, it looks like there is no magic redirect. But even in this case, how come the browser still asks for the permission to use the certificate.

GET http://workflowy.com:443/ HTTP/1.1 — this comes after the SSL Pass-Thru: CONNECT https://workflowy.com:443/. Should this mean, that this ssl connection has been closed and the browser is now connecting to http? Stange.

I don`t know how to copy ProxHTTPSProxy log.

So, did anyone around have real success with it? Do you have any warnings when you run it and don`t the sites still ask for certificates?

This was really promising, but I don`t get it or probably it no longer works. Regrets.

Quote:+++GET 17284+++
Using Proxy - 127.0.0.1:8081
CONNECT https://workflowy.com:443/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Proxy-Connection: keep-alive
Host: workflowy.com

+++SSL 17284:+++
SSL Pass-Thru: CONNECT https://workflowy.com:443/
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.4b Python/2.6.5
+++CLOSE 17284+++

+++GET 17285+++
Using Proxy - 127.0.0.1:8081
GET http://workflowy.com:443/ HTTP/1.1
Host: workflowy.com:443
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7
Cookie: sessionid=a9fdb1df5df7350dd7adaf279c18c465; _seg_id=566e7c951eb314e0%7C%3E1%7C%3E52f45a5bbe7a21e6%7C%3E1338155179735%7C%3E1338155535766
Cache-Control: max-age=0
Connection: keep-alive
Browser reload detected...

+++RESP 17285+++
HTTP/1.1 200 OK
Date: Sun, 27 May 2012 21:54:12 GMT
Server: Apache/2.2.12 (Ubuntu)
Vary: Cookie,Accept-Encoding
Cache-Control: no-cache
Content-Encoding: gzip
Content-Length: 6422
Connection: close
Content-Type: text/html; charset=utf-8
+++CLOSE 17285+++

[JJoe]

(May. 27, 2012 10:03 PM)Gravemind Wrote:  So, did anyone around have real success with it?

From what I remembered, I thought you would have less trouble than you are having. However, I had forgotten about the compiled executable.

Warnings issued would depend on the browser and settings. ProxHTTPSProxy will not warn.

Currently, I am not seeing the redirect from ProxHTTPSProxy (using old installation and browser). Am wondering if Window's update changed something...

Will read the thread and play more later.

[ProxRocks]

my fingers are crossed...

while i haven't played around with ProxHTTPSProxy for some time now, it has been my high hopes that it would become the wave of the future for anyone (ie, "us geeks") wishing to take matters into their own hands and "at their own risk" AXE the STUPID certificate-check CRAP...

the whole scheme is a crock of crap, we all know that malware sites can "buy" their own signed certificate (aka, "Certificate Authority breach"), so why propagate the MYTH that "certificates" correlate to "safety"? and it has been reported that due to the INEFFECTIVENESS, Chrome will no longer check for Certificate Revocation Lists - why waste the time to check something that is in itself "corrupt"?

i am encouraged that at least Chrome offers an "ignore certificates" startup command (i myself have requested the same from GreenBrowser during a recent auto-update bug report)...

but i digress...
the whole SSL-cert crap really does steam my corn !!!...

[JJoe]

(May. 28, 2012 12:52 AM)JJoe Wrote:  Currently, I am not seeing the redirect from ProxHTTPSProxy (using old installation and browser). Am wondering if Window's update changed something...

Will read the thread and play more later.

I forgot that I had altered my ProxHTTPSProxy install.

Now working with the header filter that I posted or with the sidki list entry. I still see the browser's mismatched certificate warning, as expected.

I will check the compiled executable next.

Did you set the browser to use ProxHTTPSProxy for https?

[JJoe]

(May. 28, 2012 02:43 AM)JJoe Wrote:  I will check the compiled executable next.

Also works but you will need to update the exe's "proxcert.pem".

http://prxbx.com/forums/showthread.php?tid=1479
or
http://proxomitron.info/files/index.html

[Gravemind]

Quote:Warnings issued would depend on the browser and settings. ProxHTTPSProxy will not warn.

Yes, but it seemed to me that it tricked browsers and decrypted on its own. So, shouldn`t there be no warnings if it`s set up for decryption?

Quote:Currently, I am not seeing the redirect from ProxHTTPSProxy (using old installation and browser). Am wondering if Window's update changed something...

Quote:Now working with the header filter that I posted or with the sidki list entry. I still see the browser's mismatched certificate warning, as expected.

Can you upload your ProxHTTPSProxy folder?

http://www.sendspace.com/

Quote:Also works but you will need to update the exe's "proxcert.pem".

Do you mean to get a new pem or change some settings in that proxy`s .py files?

My certificate is fine for 2012. I tried yours also, with the same results. Could be that my system is screwed.

I get the same result with your certificate:

Quote:Bad Request

Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

Hint: https://workflowy.com/

Apache/2.2.12 (Ubuntu) Server at workflowy.com Port 443

Something else is wrong with it.

There may be other programs of the kind. Maybe you know some to decrypt https and feed http into proxomitron?

Maybe "Wireshark"? Supressing the warnings is also great. Maybe "Squid" is able to do that, but even though there were some discussions on its seamless https, they say it feeds encrypted traffic anyway. I also found "Fiddler" among search results, but I don`t know any of its functions yet.

Some way to make Proxomitron on ssl would be great. Fortunately, they can`t host regular ads on https, but there are still some banners from their own servers and other sources. There will be even more in future.

[JJoe]

(May. 28, 2012 10:42 AM)Gravemind Wrote:  Yes, but it seemed to me that it tricked browsers and decrypted on its own. So, shouldn`t there be no warnings if it`s set up for decryption?

The Proxomitron's HalfSSL hides the https from the browser by changing links to http:// before the browser sees them.
ProxHTTPSProxy does not hide the initial https:// request from the browser. The ProxHTTPSProxy certificate may not be what the browser is expecting. Unexpected certificates may get warnings.

Apache/2.2.12 (Ubuntu) Server at workflowy.com Port 443 Wrote:Reason: You're speaking plain HTTP to an SSL-enabled server port. Instead use the HTTPS scheme to access this URL, please.

Looks like your ProxHTTPSProxy is providing the redirect but the resulting http:// request is not being converted to https:// by ProxHTTPSProxy.

Consider https://bugzilla.mozilla.org/ .
Browser sends request for https://bugzilla.mozilla.org/ to ProxHTTPSProxy.
ProxHTTPSProxy returns redirect to http://bugzilla.mozilla.org:443/ .
Browser sends http://bugzilla.mozilla.org:443/ request to Proxomitron.
Proxomitron forwards request for http://bugzilla.mozilla.org:443/ to ProxHTTPSProxy.
ProxHTTPSProxy corrects scheme and sends proper request, https://bugzilla.mozilla.org:443/ , to server.

Does the Proxomitron's log window show requests like http://bugzilla.mozilla.org:443/ and are they being sent to ProxHTTPSProxy?

(May. 28, 2012 10:42 AM)Gravemind Wrote:  There may be other programs of the kind. Maybe you know some to decrypt https and feed http into proxomitron?

(May. 28, 2012 10:42 AM)Gravemind Wrote:  Some way to make Proxomitron on ssl would be great.

The Proxomitron can decrypt and filter https. Unfortunately, certificates that the Proxomitron's old ssl routine can't understand and the unexpected certificate that the Proxomitron provides the browser cause unnecessary warnings to be issued.

The problem with providing "seamless https" is that it could be used for evil.

(May. 28, 2012 10:42 AM)Gravemind Wrote:  Can you upload your ProxHTTPSProxy folder?

(May. 28, 2012 10:42 AM)Gravemind Wrote:  Do you mean to get a new pem or change some settings in that proxy`s .py files?

New pem.


  ProxHTTPSProxy_0.4b.zip (Size: 9.82 KB / Downloads: 774)

[Gravemind]

I made it work finally. Though, it still asks for certificates and fails logins on many https sites like "Google".

Also, it kills by URL with \k on decrypted pages with no problem, but the web page filters behave erratically. Sometimes they work, but sometimes they don`t, and I don`t know how to explain that.

Anyway, it is better than nothing.

This does help a lot, even if you think you know how it works:

Quote:Consider https://bugzilla.mozilla.org/ .
Browser sends request for https://bugzilla.mozilla.org/ to ProxHTTPSProxy.
ProxHTTPSProxy returns redirect to http://bugzilla.mozilla.org:443/ .
Browser sends http://bugzilla.mozilla.org:443/ request to Proxomitron.
Proxomitron forwards request for http://bugzilla.mozilla.org:443/ to ProxHTTPSProxy.
ProxHTTPSProxy corrects scheme and sends proper request, https://bugzilla.mozilla.org:443/ , to server.

Although, I don`t fully understand what happens after the last step. It receives https, decrypts and sends the http back into the Proxomitron?

The funny thing is that it is still asking for permission and there is the initial warning to add the cert. to exceptions. Since the traffic is supposed to be already decrypted by openssl and the like, and the browser hasn`t sent https request (there was a redirect), the warnings must be suppressed. It is still bugging me that it is coming from the browser while the url bar is not locked. But I may not see the whole picture.

Quote:Does the Proxomitron's log window show requests like http://bugzilla.mozilla.org:443/ and are they being sent to ProxHTTPSProxy?

Quote:+++GET 3884+++
GET /ALFAIBSR/ HTTP/1.1
Host: click.alfabank.ru
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7
Cookie: cookie=on; auth.uid=46.237.0.24%3ANULL%3D16890cc90999dab5ff420dfe3337eebd; auth.uid.updated=2012-05-24%2018%3A39%3A21
Connection: keep-alive

+++RESP 3884+++
HTTP/1.1 307 Temporary Redirect
Location: https://click.alfabank.ru/ALFAIBSR/
Connection: close
Content-Type: text/html; charset=UTF-8
Content-Length: 171
+++CLOSE 3884+++

+++GET 3885+++
Using Proxy - 127.0.0.1:8081
GET http://click.alfabank.ru:443/ALFAIBSR/ HTTP/1.1
Host: click.alfabank.ru:443
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru-ru,ru;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7
Cookie: cookie=on; auth.uid=46.237.0.24%3ANULL%3D16890cc90999dab5ff420dfe3337eebd; auth.uid.updated=2012-05-24%2018%3A39%3A21
Connection: keep-alive

I have set up ProxHTTPSProxy in the browser SSL-proxy settings, without using any commands in Proxom. headers (not really, I have one rule for the Proxomitron to send http://...:443/ — type requests).

That is why I only see this info when I input the https address like http.

Quote:The problem with providing "seamless https" is that it could be used for evil.

Yeah, but somehow that totally doesn`t worry me. I found there are entire companies spying on their folks with "man-in-the-middle" proxies. It only took them to import the CA-root certificate for those proxies like "Squid" and "Wireshark" for everybody to notice nothing. It is nice when you can decrypt it for your own use. Like for Proxomitron.

Quote:ProxHTTPSProxy_0.4b.zip (Size: 9.82 KB / Downloads: 4)

When I first launched your file, it chunked and shut down. I then tried the old files from the first message here, they also died. I then uninstalled python 3+, restarted and installed python 2+. And it started working.

Also, I found 2 interesting proxy projects. At the moment, I don`t have the skills to turn them into useful gateways for the Proxomitron. Maybe I could cope with the second one, but the first is only good for its source code which I can`t do much with.

Here they are:

http://code.google.com/p/proxpy/

Quote:ProxPy is a highly customizable HTTP/HTTPS proxy, written in Python. It is very handy for web penetration testers and for developers interested in testing their web applications.

ProxPy works as a "man-in-the-middle" between the browser and the target application. It has been developed with the purpose to be easily customizable. At this aim, users can write plug-in with minimal effort. Plug-ins are written in Python, and can modify HTTP/HTTPS requests and response on-the-fly.

Please note that ProxPy is currently under heavy development, so the plug-ins interface may change in the near future.

https://github.com/nodejitsu/node-http-proxy

Quote:Using HTTPS

You have all the full flexibility of node-http-proxy offers in HTTPS as well as HTTP. The two basic scenarios are: with a stand-alone proxy server or in conjunction with another HTTPS server.
Proxying to HTTP from HTTPS

This is probably the most common use-case for proxying in conjunction with HTTPS. You have some front-facing HTTPS server, but all of your internal traffic is HTTP. In this way, you can reduce the number of servers to which your CA and other important security files are deployed and reduce the computational overhead from HTTPS traffic.

Imagine if it does what it says. Would be nice.

I will be posting a detailed manual on this helper program below. For "noobs".

[Gravemind]

ProxHTTPSProxy — a Proxomitron SSL Helper Program — For dummies

First, download this:

http://prxbx.com/forums/attachment.php?aid=762

This is from a few posts above.

Then:

1. Go to http://www.python.org/download/

Download and install:

Python 2.7.3 Windows Installer (Windows binary -- does not include source)

or

Python 2.7.3 Windows X86-64 Installer (Windows AMD64 / Intel 64 / X86-64 binary [1] -- does not include source)

2. Go to http://slproweb.com/products/Win32OpenSSL.html and download Win32 OpenSSL v1.0.1c — 16MB Installer (worked for me, I don`t know which one is good for x64 — you try it). Install. Restart PC.

3.Go to "Firefox" settings and set this in the proxy section (others — network — connection — settings / my version is not in English, this is just an approximation):

check "Manual proxy settings"
HTTP-proxy: localhost on 8192 (could differ in your Proxomitron)
SSL-proxy: localhost on 8081

Then, launch ProxHTTPSProxy (from the locally downloaded folder), you should see a black window.

Open Proxomitron > Proxy > Add > 127.0.0.1:8081 > OK.

You can test it. It should post OK in the log window, in both of them.

Then go to proxomitron Headers and import this rule:

Quote:[HTTP headers]
In = FALSE
Out = TRUE
Key = "ProxHTTPSProxy"
URL = "$URL(http://?++:443/\w) $SETPROXY(127.0.0.1:8081)"

Now it`s ready. Test it on any https website. You will see a warning. Make an exception for the certificate if you want to filter it.

The filtering is erratic for me. It does kill urls with \k, also some of the tag filters are fine. But the https google search page is totally random. Sometimes it filters all ads in it, then it doesn`t. If you know the reason for this, make a post.

To test your setup import this rule into Web page

Quote:[Patterns]
Name = "TEST"
Active = TRUE
URL = "\wworkflowy.com"
Limit = 20
Match = "id="billboard""
Replace = "style="display:none""

https://workflowy.com/

Go to this site and the login area must be erased.

Congratulations!

NOTE: In order to control the https filtering in a flexible way — switch it on and off only with the Bypass button, do not set ssl-proxy in your browser settings, but import this rule into Proxomitron:

Quote:[HTTP headers]
In = FALSE
Out = TRUE
Key = "ProxHTTPSProxy 2"
URL = "$URL(https://?+) $SETPROXY(127.0.0.1:8081)"

The SSL-proxy in your browser must be the Proxomitron for this to work.

[Gravemind]

I forgot the main part.

Use "WinRoll", "4t Tray Minimizer Free" or any other program to hide the ugly black window in the notification area of the task bar or somewhere else.

[Gravemind]

Also, you must have the proxcert.pem from the ProxHTTPSProxy imported into your browsers decryption storage.

It does not guarantee that you will get https working on all website you want.

Https decryption in the Proxomitron should be off.

[JJoe]

(May. 28, 2012 09:08 PM)Gravemind Wrote:  fails logins on many https sites like "Google".

There can be more to it than hiding https from the browser. You may need to alter other headers like set-cookie and referer.

(May. 28, 2012 09:08 PM)Gravemind Wrote:  Although, I don`t fully understand what happens after the last step. It receives https, decrypts and sends the http back into the Proxomitron?

Correct.

I don't wish to confuse things but can you now see that it is possible to have the browser send both http and https to the Proxomitron and still have the Proxomitron use ProxHTTPSProxy for https?

(May. 28, 2012 09:08 PM)Gravemind Wrote:  The funny thing is that it is still asking for permission and there is the initial warning to add the cert. to exceptions. Since the traffic is supposed to be already decrypted by openssl and the like, and the browser hasn`t sent https request (there was a redirect), the warnings must be suppressed. It is still bugging me that it is coming from the browser while the url bar is not locked. But I may not see the whole picture.

The browser tries to initiate a secure connection with "bugzilla.mozilla.org". Our local proxy steps in and provides its credentials. Unfortunately or fortunately, these credentials (proxcert.pem) do not identify the local proxy as "bugzilla.mozilla.org" but as "Proxomitron". So, user input may be required before the browser will allow a secure connection to be established and data to be sent.

(May. 28, 2012 09:08 PM)Gravemind Wrote:  Yeah, but somehow that totally doesn`t worry me. I found there are entire companies spying on their folks with "man-in-the-middle" proxies. It only took them to import the CA-root certificate for those proxies like "Squid" and "Wireshark" for everybody to notice nothing. It is nice when you can decrypt it for your own use. Like for Proxomitron.

I think it is more difficult to create a remote mitm than you think. Especially since some companies were found to be issuing certificates to allow snooping. Removing or hiding warnings on the local machine would make it easy, however. Again, I think.

(May. 28, 2012 09:08 PM)Gravemind Wrote:  I then uninstalled python 3+, restarted and installed python 2+. And it started working.

Ah, python 3 series actually broke some things, iirc. Apparently, not fixed yet.

Does the compiled executable, http://proxfilter.net/ProxHTTPSProxy.zip , work for you?