ProxHTTPSProxyMII: Development

[JJoe]

(Jul. 13, 2014 02:57 PM)GunGunGun Wrote:  And now I have a small question, can ProxHTTPSProxy use certs.pem together with proxcert.pem ?

Are you using the original ProxHTTPSProxy or ProxHTTPSProxyMII

The original ProxHTTPSProxy does not check certificates. So "certs.pem" is not used.

The files that ProxHTTPSProxyMII needs, "CA.crt" and "cacert.pem", are in the ProxHTTPSProxyMII zip.

[GunGunGun]

(Jul. 13, 2014 04:49 PM)JJoe Wrote:  

(Jul. 13, 2014 02:57 PM)GunGunGun Wrote:  And now I have a small question, can ProxHTTPSProxy use certs.pem together with proxcert.pem ?

Are you using the original ProxHTTPSProxy or ProxHTTPSProxyMII

The original ProxHTTPSProxy does not check certificates. So "certs.pem" is not used.

The files that ProxHTTPSProxyMII needs, "CA.crt" and "cacert.pem", are in the ProxHTTPSProxyMII zip.

I'm using ProxHTTPSProxy, I tried to use ProxHTTPSProxyMII anyway but cannot make this version work, everytime I open ProxHTTPSProxyMII this message show:


And I don't know why everytime I try to connect to addons.mozilla.org or facebook.com this message show:



  2014-07-14_081742.jpg (Size: 91.18 KB / Downloads: 833)
  2014-07-14_081934.jpg (Size: 92.83 KB / Downloads: 762)

A new and fresh Firefox Portable profile, but still get the error:
  2014-07-14_082725.jpg (Size: 107.25 KB / Downloads: 899)
ProxHTTPSProxy 0.4:
  2014-07-14_082840.jpg (Size: 219.57 KB / Downloads: 794)

But works really well with Opera or Chrome:
  2014-07-14_085025.jpg (Size: 198.76 KB / Downloads: 902)

The "I understand the risk" was added by Skip Cert Error addons, but probably do nothing, I tried to Add Exception but still show this message, do you have a workaround to fix this problem ?

I'm using Firefox 30. I found this thread, do every workaround (change Firefox about:config, even generate a new proxcert.pem file) at this thread but still happen, but only with Firefox, Chrome and Opera work well, and can open addons.mozilla.org and facebook.com. But I don't want to use Opera or Chrome at all, Firefox for me is the best browser with low CPU and RAM cost, I also have a bunch tweak with the browser too.

[JJoe]

(Jul. 14, 2014 01:24 AM)GunGunGun Wrote:  The "I understand the risk" was added by Skip Cert Error addons, but probably do nothing, I tried to Add Exception but still show this message, do you have a workaround to fix this problem ?

Firefox does not like self-signed certs. The "Skip Cert Error addon" is used to automatically add an exemption to dismiss the "This Connection is Untrusted" warning from Firefox.

When "Skip Cert Error addon" works, you should not see the "This Connection is Untrusted" warning. However, the addon only 'works' for the files shown in the browser's window.

Your images seem to show that "Skip Cert Error addon" is not installed or 'working'.

In "Skip Cert Error addon" options did you enable all under "Bypass error when cert:"?

https://addons.mozilla.org/en-US/firefox...ert-error/

(Jul. 14, 2014 01:24 AM)GunGunGun Wrote:  I'm using ProxHTTPSProxy, I tried to use ProxHTTPSProxyMII anyway but cannot make this version work, everytime I open ProxHTTPSProxyMII this message show:

Basic installation instructions for all proxies should be the same.

For ProxHTTPSProxyMII exe version, did you:

Code:

Install Win32OpenSSL_Light-1_0_1h. OpenSSL needs "Visual C++ 2008 Redistributables".

Add ProxHTTPSProxy's "CA.crt" to the browser's store of trusted certificate authorities.
Set the browser to use the ProxHTTPSProxy front server for secure connections.
      The front server's default address is 127.0.0.1 on port 8079.
Set the filtering proxy to receive requests from the front server.
      Default address for the filtering proxy is 127.0.0.1 on port 8080.
Set the filtering proxy to forward requests to the ProxHTTPSProxy rear server.
      The rear server's default address is 127.0.0.1 on port 8081.
Execute ProxHTTPSProxy.exe to start.

?

I don't have enough experience with "PortableApp" to know what issues it might cause.

HTH

[GunGunGun]

(Jul. 14, 2014 06:05 PM)JJoe Wrote:  

(Jul. 14, 2014 01:24 AM)GunGunGun Wrote:  The "I understand the risk" was added by Skip Cert Error addons, but probably do nothing, I tried to Add Exception but still show this message, do you have a workaround to fix this problem ?

Firefox does not like self-signed certs. The "Skip Cert Error addon" is used to automatically add an exemption to dismiss the "This Connection is Untrusted" warning from Firefox.

When "Skip Cert Error addon" works, you should not see the "This Connection is Untrusted" warning. However, the addon only 'works' for the files shown in the browser's window.

Your images seem to show that "Skip Cert Error addon" is not installed or 'working'.

In "Skip Cert Error addon" options did you enable all under "Bypass error when cert:"?

Yes, I did, Skip Cert installed correctly but seem problem maybe half-SSL cannot give me full ability to access some SLL sites like addons.mozilla.org, I tried MII then works, but I can only make MII works with Proxomitron, I don't know how to make MII works with Privoxy... I will try more experiment and I will share my experiment when I success.
Seem I can only make MII help Privoxy filter site like Google.com, but cannot filter site like addons.mozilla.org, my setup:

Code:

My Firefox already had CA.crt from MII
I set my Firefox HTTP Proxy to 127.0.0.1:8118 - Privoxy port
and HTTPS Proxy to 127.0.0.1:8079 - MII port

Privoxy user.action:

Code:

{+forward-override{forward 127.0.0.1:8081}}
:443

And config.ini from MII:

Code:

[GENERAL]
ProxAddr = http://127.0.0.1:8118
FrontPort = 8079
RearPort = 8081

# The proxy has to support HTTPS CONNECT
#DefaultProxy = https://127.0.0.1:8118

# Proper values for LogLevel are ERROR, WARNING, INFO, DEBUG
# Default is INFO if unset
LogLevel =

[OPENSSL]
PATH = C:\OpenSSL-Win32\bin\openssl.exe

# * matches everything
# ? matches any single character
# [seq] matches any character in seq
# [!seq] matches any character not in seq

### Bypass Proxomitron, Proxy setting still effective
[SSL Pass-Thru]
pypi.python.org

[PROXY https://192.168.1.38:8123]
duckduckgo.com
abc.net

[PROXY https://127.0.0.1:8124]
www.test2.com
abc.org

# Socks proxy support
# https://github.com/shazow/urllib3/pull/284

Quote:https://addons.mozilla.org/en-US/firefox...ert-error/

(Jul. 14, 2014 01:24 AM)GunGunGun Wrote:  I'm using ProxHTTPSProxy, I tried to use ProxHTTPSProxyMII anyway but cannot make this version work, everytime I open ProxHTTPSProxyMII this message show:

Basic installation instructions for all proxies should be the same.

For ProxHTTPSProxyMII exe version, did you:

Code:

Install Win32OpenSSL_Light-1_0_1h. OpenSSL needs "Visual C++ 2008 Redistributables".

Add ProxHTTPSProxy's "CA.crt" to the browser's store of trusted certificate authorities.
Set the browser to use the ProxHTTPSProxy front server for secure connections.
      The front server's default address is 127.0.0.1 on port 8079.
Set the filtering proxy to receive requests from the front server.
      Default address for the filtering proxy is 127.0.0.1 on port 8080.
Set the filtering proxy to forward requests to the ProxHTTPSProxy rear server.
      The rear server's default address is 127.0.0.1 on port 8081.
Execute ProxHTTPSProxy.exe to start.

?

I don't have enough experience with "PortableApp" to know what issues it might cause.

HTH

Thank you, I tried and MII works!

[whenever]

@GunGunGun, I don't know Privoxy but you need to find a way to forward all requests with "Tagged:Proxomitron FrontProxy/*" header to the ProxHTTPSProxyMII rear server. The alternate way is to run 2 instances of Privoxy, one for general http and another for ProxHTTPSProxyMII only.

(Jul. 08, 2014 04:41 AM)JJoe Wrote:  However, should you see a quick, easy, and reasonable way to add a custom header to tag files that have been clicked on or actually requested by the user... please share.

Do you have more ideas about this?

BTW, I'll have only limited computer access during August 1st-20th. It would be great if we can package a public version before that.

[JJoe]

(Jul. 16, 2014 07:31 AM)whenever Wrote:  

(Jul. 08, 2014 04:41 AM)JJoe Wrote:  However, should you see a quick, easy, and reasonable way to add a custom header to tag files that have been clicked on or actually requested by the user... please share.

Do you have more ideas about this?

None that are quick, easy, or reasonable. lol

The Proxomitron can watch for user keypress but it does not know why the user pressed the key.

(Jul. 16, 2014 07:31 AM)whenever Wrote:  BTW, I'll have only limited computer access during August 1st-20th. It would be great if we can package a public version before that.

Am trying. I want to use it for the monthly prox-list post for July.

[GunGunGun]

Hi JJoe and whenever, I finally found a way to make ProxHTTPSProxy works with Privoxy, here is my instruction:

Code:

- First open Privoxy folder, open user.filter and add:

CLIENT-HEADER-TAGGER: HTTPSTAG
s@^.*Tagged:.*Proxomitron FrontProxy.*$@$0@img

- Open user.action add:
{+client-header-tagger{HTTPSTAG}}
/

{+forward-override{forward 127.0.0.1:8081}}
TAG:.*Proxomitron

- Import CA.crt to your browser Certificate manager.

- Open ProxHTTPSProxy folder and change ProxAddr = http://localhost:8080 to ProxAddr = http://localhost:8118

Set your browser proxy to HTTP 127.0.0.1:8118 HTTPS: 127.0.0.1:8079 and you are all done, now Privoxy can filter HTTPS website, even addons.mozilla.org.

If you want to use MII with Python portable, simply copy you Python folder to usb, use this bat file to open MII:

Code:

python.exe "ProxHTTPSProxy.py"

Probably you need to install pyOpenSSL, url3lib and colorama using pip install from the Scripts folder first or latter.


__________________________________EDIT______________________________

I want to report a bug that ProxHTTPSProxyMII won't work on this page: https://rapidgator.net/auth/login
Error message:

Code:

Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598).

[whenever]

(Jul. 19, 2014 02:49 AM)GunGunGun Wrote:  I want to report a bug that ProxHTTPSProxyMII won't work on this page: https://rapidgator.net/auth/login

The shipped cacert.pem is not complete. You can open it with a text editor and add below lines to its end, then you are good to go.

Code:

# RapidSSLCA
-----BEGIN CERTIFICATE-----
MIID1TCCAr2gAwIBAgIDAjbRMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMTAwMjE5MjI0NTA1WhcNMjAwMjE4MjI0NTA1WjA8MQswCQYDVQQG
EwJVUzEXMBUGA1UEChMOR2VvVHJ1c3QsIEluYy4xFDASBgNVBAMTC1JhcGlkU1NM
IENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAx3H4Vsce2cy1rfa0
l6P7oeYLUF9QqjraD/w9KSRDxhApwfxVQHLuverfn7ZB9EhLyG7+T1cSi1v6kt1e
6K3z8Buxe037z/3R5fjj3Of1c3/fAUnPjFbBvTfjW761T4uL8NpPx+PdVUdp3/Jb
ewdPPeWsIcHIHXro5/YPoar1b96oZU8QiZwD84l6pV4BcjPtqelaHnnzh8jfyMX8
N8iamte4dsywPuf95lTq319SQXhZV63xEtZ/vNWfcNMFbPqjfWdY3SZiHTGSDHl5
HI7PynvBZq+odEj7joLCniyZXHstXZu8W1eefDp6E63yoxhbK1kPzVw662gzxigd
gtFQiwIDAQABo4HZMIHWMA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUa2k9ahhC
St2PAmU5/TUkhniRFjAwHwYDVR0jBBgwFoAUwHqYaI2J+6sFZAwRfap9ZbjKzE4w
EgYDVR0TAQH/BAgwBgEB/wIBADA6BgNVHR8EMzAxMC+gLaArhilodHRwOi8vY3Js
Lmdlb3RydXN0LmNvbS9jcmxzL2d0Z2xvYmFsLmNybDA0BggrBgEFBQcBAQQoMCYw
JAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmdlb3RydXN0LmNvbTANBgkqhkiG9w0B
AQUFAAOCAQEAq7y8Cl0YlOPBscOoTFXWvrSY8e48HM3P8yQkXJYDJ1j8Nq6iL4/x
/torAsMzvcjdSCIrYA+lAxD9d/jQ7ZZnT/3qRyBwVNypDFV+4ZYlitm12ldKvo2O
SUNjpWxOJ4cl61tt/qJ/OCjgNqutOaWlYsS3XFgsql0BYKZiZ6PAx2Ij9OdsRu61
04BqIhPSLT90T+qvjF+0OJzbrs6vhB6m9jRRWXnT43XcvNfzc9+S7NIgWW+c+5X4
knYYCnwPLKbK3opie9jzzl9ovY8+wXS7FXI6FoOpC+ZNmZzYV+yoAVHHb1c0XqtK
LEL2TxyJeN4mTvVvk0wVaydWTQBUbHq3tw==
-----END CERTIFICATE-----

[whenever]

In case anyone gets problem with the latest Firefox v31, please use attached CA.crt instead.

Remember to remove the old CA from the trusted store and empty the certs folder.

BTW, I got quite frequently below message when reply.

Quote:The maximum server load limit has been reached. Please check back later once the server is less busy.

[JJoe]

(Jul. 17, 2014 04:08 AM)JJoe Wrote:  

(Jul. 16, 2014 07:31 AM)whenever Wrote:  BTW, I'll have only limited computer access during August 1st-20th. It would be great if we can package a public version before that.

Am trying. I want to use it for the monthly prox-list post for July.

There was an unexpected illness, death, and funeral. I'm just now getting back to other things.

So, don't expect days of my best work...

[whenever]

JJoe, I am sorry for your lose.

Don't push yourself hard. Perfect is the enemy of good.

Attached would be the ver 1.0 if there are no obvious problems found. It adds blacklist and Certificate no-check list features, and has various minor fixes.

[JJoe]

Better or worse?

An assist to dillinger. An example Markdown file follows but I modified the resulting html while spending too much time debugging Internet Explorer. A problem was Windows knew that I had got the html files from the web and had blocked some but not all local resources. I disabled blocking when checking the properties of the files. Also, the dillinger html files where unix formated.

Code:

ProxHTTPSProxyMII
====

Created to provide modern nag-free HTTPS connections for an HTTP proxy.

```
Client HTTPS <-> ProxHTTPSProxyFront <-> HTTP Proxy <-> ProxHTTPSProxyRear <-> Server HTTPS
```

Eligible HTTP Proxies
----
* The [Proxomitron], for which ProxHTTPSProxy was created :)
* Any that have the ability to forward all requests with a "Tagged:Proxomitron FrontProxy/*" header to the ProxHTTPSProxyMII rear server.
* Any that can be ran as two instances, one for true http and another for "tagged" http
* Any that will only be used to monitor https traffic  

Install
----
* [Python 3.4] to c:\Python34
* [Win32OpenSSL_Light-1_0_1h] (OpenSSL needs "[Visual C++ 2008 Redistributables]")
* [pyOpenSSL]
* [urllib3]
* [colorama]
* ProxHTTPSProxy's "CA.crt" to the Client's store of trusted certificate authorities.

Configure
----
* The Client to use the ProxHTTPSProxy front server at 127.0.0.1 on port 8079 for secure connections.
* The HTTP proxy to receive requests at 127.0.0.1 on port 8080.
* The HTTP proxy to forward requests to the ProxHTTPSProxy rear server at 127.0.0.1 on port 8081.
* Edit "Config.ini" to change these requirements.

Execute
----
ProxHTTPSProxy.py to start.

Remember
----
Be aware and careful! Use a direct connection when you don't want any mistakes made.

Use at your own risk!

Have fun!

[Proxomitron]:http://www.proxomitron.info
[Python 3.4]:https://www.python.org/download/
[Win32OpenSSL_Light-1_0_1h]:http://www.slproweb.com/products/Win32OpenSSL.html
[Visual C++ 2008 Redistributables]:http://www.slproweb.com/products/Win32OpenSSL.html
[pyOpenSSL]:https://pypi.python.org/pypi/pyOpenSSL/0.14
[urllib3]:https://pypi.python.org/pypi/urllib3
[colorama]:https://pypi.python.org/pypi/colorama

[whenever]

It's great. Based on your work, I published the exe version 1.0 at http://prxbx.com/forums/showthread.php?tid=2172

I would like to leave this thread for development and that thread for users discuss.

I know it is not perfect yet but perfect is the enemy of good, so let's just get the ball rolling!

[GunGunGun]

Thank you.

Small note: If anyone wants to use ProxHTTPSProxy with Python Portable then go to C:\Python34\Lib and copy configparser.py into your ProxHTTPSProxy 1.0+ folder, only 1.0+ have this problem.

Tutorial how to make ProxHTTPSProxy works with Privoxy and also provided a way to use ProxHTTPSProxy with Python Portable: http://prxbx.com/forums/showthread.php?t...7#pid17677

_____________________

Hi whenever, can you add a feature that make ProxHTTPSProxy auto generate new certificate for some site like rapidgator.net, probably user have to Accept the new certificate by clicking something like a confirm dialog ? http://prxbx.com/forums/showthread.php?t...8#pid17678

For me, if I meet some site like Rapidgator again then I think there is no way that I can manually generate a new certificate to make that site works.

______________________

Hi, I edit this post because I really want to request a new feature for ProxHTTPSProxy:

About blacklist feature, can you make this feature can block domain + path rule, like Proximitron or Privoxy already did ? Example:
[BLACKLIST]
facebook.com/ads

I tried but it doesn't work, nothing get blocked.

[whenever]

(Aug. 27, 2014 02:03 PM)GunGunGun Wrote:  Hi whenever, can you add a feature that make ProxHTTPSProxy auto generate new certificate for some site like rapidgator.net, probably user have to Accept the new certificate by clicking something like a confirm dialog ? http://prxbx.com/forums/showthread.php?t...8#pid17678

For me, if I meet some site like Rapidgator again then I think there is no way that I can manually generate a new certificate to make that site works.

That would defeat the security provided by CA chains. However, you can do it manually if you decide to trust it. Major browsers have a "view certificate" function where you can export the CA to a file then you can append the content to ProxHTTPSProxyMII's cacert.pem file.

(Aug. 27, 2014 02:03 PM)GunGunGun Wrote:  About blacklist feature, can you make this feature can block domain + path rule, like Proximitron or Privoxy already did ?

I prefer to keep this program as simple as possible. I think Proxomitron and Privoxy can do URL blocking better.

BTW, I don't have time to work on this program any more until October. I just hope I hadn't forgot Python yet by then.