Post Reply 
ProxHTTPSProxy, a Proxomitron SSL Helper Program
May. 20, 2010, 08:23 AM
Post: #16
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 20, 2010 07:54 AM)JJoe Wrote:  
Code:
File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------

I have some 3rd party modules installed on top of python 2.6.5 but that should not be a problem because they are not used in this program.

The SSLError comes from python's standard library and seems related to the certificate file. Please test with the attached proxcert.pem file I am currently use.


Attached File(s)
.zip  proxcert.zip (Size: 1.39 KB / Downloads: 881)
Add Thank You Quote this message in a reply
May. 20, 2010, 12:50 PM
Post: #17
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 20, 2010 08:23 AM)whenever Wrote:  Please test with the attached proxcert.pem file I am currently use.

No joy. Result is Exception call.

Code:
Exception happened during processing of request from

I'll play with it later.
Add Thank You Quote this message in a reply
May. 20, 2010, 05:27 PM
Post: #18
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
The errors that JJoe has shown look strikingly similar to: http://stackoverflow.com/questions/26300...ertificate

The circumstances are different with a PEM vs. PKCS#12 cert, yet it's happening at the same lines of the Python library. I wonder if the correct 'proxcert.pem' is being found & loaded.
Add Thank You Quote this message in a reply
May. 20, 2010, 11:02 PM
Post: #19
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 20, 2010 12:50 PM)JJoe Wrote:  I'll play with it later.

Will I need anything from http://www.slproweb.com/products/Win32OpenSSL.html ?
I don't have Visual C++ 2008 Redistributables or Shining Light Productions OpenSSL installed.

There is also a PyOpenSSL, http://pypi.python.org/pypi/pyOpenSSL .

I downloaded Tiny HTTP Proxy in Python from http://www.okisoft.co.jp/esc/python/proxy/ . It worked with Proxo, so part of my install is possibly ok.

WebCleaner requires a number of things. It can filter https.
http://webcleaner.sourceforge.net/install.html
Add Thank You Quote this message in a reply
May. 21, 2010, 01:30 AM
Post: #20
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 20, 2010 11:02 PM)JJoe Wrote:  Will I need anything from http://www.slproweb.com/products/Win32OpenSSL.html ?
I don't have Visual C++ 2008 Redistributables or Shining Light Productions OpenSSL installed.

That might be the point. My OpenSSL copy was installed before I play with python. Sorry I didn't notice python SSL module require it.

From the python doc:

Quote:This SSL module uses the OpenSSL library. It is available on all modern Unix systems, Windows, Mac OS X, and probably additional platforms, as long as OpenSSL is installed on that platform.
Add Thank You Quote this message in a reply
May. 21, 2010, 01:53 AM
Post: #21
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 01:30 AM)whenever Wrote:  Sorry I didn't notice python SSL module require it.

Don't worry. Smile!
These things are expected.

I'll get it installed.
Cross your fingers. Wink
Add Thank You Quote this message in a reply
May. 21, 2010, 05:43 AM
Post: #22
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 20, 2010 11:02 PM)JJoe Wrote:  WebCleaner requires a number of things. It can filter https.
http://webcleaner.sourceforge.net/install.html
wow! i've not seen this yet, looks VERY promising!
Add Thank You Quote this message in a reply
May. 21, 2010, 08:27 AM (This post was last modified: May. 23, 2010 07:18 PM by JJoe.)
Post: #23
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 01:53 AM)JJoe Wrote:  I'll get it installed.
Cross your fingers. Wink

An update before I get some sleep.

The exception/error appears to have been fixed by installing Win32OpenSSL-0_9_8n. However, I uninstalled everything and reinstalled and added some.

Code:
ProxHTTPSProxy/0.1 serving on localhost:8081, <Ctrl-C> to stop ...
E3-PC - - [21/May/2010 02:10:19] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [21/May/2010 02:10:20] code 304, message Not Modified
E3-PC - - [21/May/2010 02:10:20] "GET http://ssl.scroogle.org:443/ HTTP/1.1" 304
-
E3-PC - - [21/May/2010 02:11:22] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.E03-PC - - [21/May/20
10 02:11:22] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
.0.1', 56223)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 118, in __init__
    self.do_handshake()
  File "C:\Python26\lib\ssl.py", line 293, in do_handshake
    self._sslobj.do_handshake()
SSLError: [Errno 6] _ssl.c:480: TLS/SSL connection has been closed
----------------------------------------
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 56224)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 60, in do_CONNECT
    path = ssl_sock.recv(512).split(None, 2)[1]
IndexError: list index out of range
----------------------------------------
E3-PC - - [21/May/2010 02:11:24] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [21/May/2010 02:11:25] "GET http://ssl.scroogle.org:443/ HTTP/1.1" 200

I think it works.
Edit: I can't be sure it worked,
http://prxbx.com/forums/showthread.php?t...4#pid14244 .

Browser HTTP is sent to The Proxomitron at 8080
Browser HTTPS is sent to ProxHTTPSProxy at 8081
In sidki's Exceptions-U.ptxt
Code:
# ProxHTTPSProxy
[^/]++:&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

So with "ssl.scroogle.org[:/]", Proxo always had a https address to forward to ProxHTTPSProxy and we loop. The new entry only forwards http.

No big list or tags needed. A fairly slick FrankenProxy.
I'm seeing some exceptions. The... I think you'll get there.

To bed.

Have fun
Add Thank You Quote this message in a reply
May. 21, 2010, 08:59 AM
Post: #24
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 05:43 AM)ProxRocks Wrote:  
(May. 20, 2010 11:02 PM)JJoe Wrote:  WebCleaner requires a number of things. It can filter https.
http://webcleaner.sourceforge.net/install.html
wow! i've not seen this yet, looks VERY promising!

The 2.41 version in the download section won't install with python 2.6. The change log said there was a 2.42 version which uses python 2.6 but there is nowhere to download the windows build. Check its Git repository you can see it is still actively developed.

There is another filtering proxy you might be interested. http://middle-man.sourceforge.net/

The delegate project also has the filtering ability.

In the end, the XUProxy is what I expected mostly. Wink
Add Thank You Quote this message in a reply
May. 21, 2010, 02:41 PM
Post: #25
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 08:27 AM)JJoe Wrote:  I think it works.

Great! Cheers

(May. 21, 2010 08:27 AM)JJoe Wrote:  In sidki's Exceptions-U.ptxt
Code:
# ProxHTTPSProxy
[^/]++:&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

This entry will catch http://www.NotHttpsSite.com:8080/ too, which will cause problem.

Except for that, maybe we'd better use "ssl.scroogle.org:" with it?

If a site uses https for login or some other thing then switch back to http. "www.site.com[:/]" will cause problem on http pages.

(May. 21, 2010 08:27 AM)JJoe Wrote:  I'm seeing some exceptions. The... I think you'll get there.

Here is a slightly updated version. I am not going to handle some exceptions raised by python's standard library. Just leave them there for information.


Attached File(s)
.zip  ProxHTTPSProxy 0.1a.zip (Size: 1.35 KB / Downloads: 878)
Add Thank You Quote this message in a reply
May. 21, 2010, 11:16 PM (This post was last modified: May. 23, 2010 07:18 PM by JJoe.)
Post: #26
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 02:41 PM)whenever Wrote:  
(May. 21, 2010 08:27 AM)JJoe Wrote:  I think it works.

Great! Cheers

Now it doesn't work. Sad

Code:
ProxHTTPSProxy/0.1a serving on localhost:8081, <Ctrl-C> to stop ...
E3-PC - - [21/May/2010 16:43:46] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.
E3-PC - - [21/May/2010 16:43:47] "CONNECT ssl.scroogle.org:443 HTTP/1.0" 200 -
failed to recv client request info, try again.

The Proxomitron's log window also looks different now.
Edit: http://prxbx.com/forums/showthread.php?t...4#pid14244 .

Success -> turn off computer -> turn on computer -> Failure
Proxo has done this to me before. Darn...

(May. 21, 2010 02:41 PM)whenever Wrote:  This entry will catch http://www.NotHttpsSite.com:8080/ too, which will cause problem.

Except for that, maybe we'd better use "ssl.scroogle.org:" with it?

If a site uses https for login or some other thing then switch back to http. "www.site.com[:/]" will cause problem on http pages.

The only time I had this work (it doesn't work now),
Opera's request for https://www.site.com/ went to ProxHTTPSProxy
ProxHTTPSProxy responded 307 http://www.site.com:443/
Opera's request for http://www.site.com:443/ went to The Proxomitron
The Proxomitron forwarded this request to ProxHTTPSProxy

ProxHTTPSProxy 'tagged' the url with :443. Most of the addresses that I see don't use port number.

[^/]++:443&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)
would catch and forward most (almost all?) requests 'tagged' by ProxHTTPSProxy, I think. A small list might handle exceptions.

You might be able to use a cookie for a tag.

Again, I think.
Add Thank You Quote this message in a reply
May. 22, 2010, 02:01 AM
Post: #27
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 21, 2010 11:16 PM)JJoe Wrote:  
Code:
E3-PC - - [21/May/2010 16:43:46] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
failed to recv client request info, try again.
failed to recv client request info, try again.

This is normal. The prompt is there for information.

ProxHTTPSProxy reads the request url after the SSL tunnel is established before it can issue a 307 response. Sometimes it is too fast and the client hasn't send the info yet so I use a loop to read forever until it gets the info.

(May. 21, 2010 11:16 PM)JJoe Wrote:  Success -> turn off computer -> turn on computer -> Failure
Proxo has done this to me before. Darn...

Now I feel relieved it might not be my program's problem. Smile!

(May. 21, 2010 11:16 PM)JJoe Wrote:  ProxHTTPSProxy 'tagged' the url with :443. Most of the addresses that I see don't use port number.

[^/]++:443&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)
would catch and forward most (almost all?) requests 'tagged' by ProxHTTPSProxy, I think. A small list might handle exceptions.

I like this way. Smile!
Add Thank You Quote this message in a reply
May. 22, 2010, 02:19 AM
Post: #28
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 22, 2010 01:58 AM)Graycode Wrote:  In this I tried to address what I think is the issue you encountered above. The changes I made start at line# 63.

The mod reads character by character. I saw them in other python proxy's code too. In my code I read up to 1024 bytes a time because it takes only 1 line and looks simple.

Graycode, what's the difference between reading character by character and reading a bunch from a socket?

JJoe, if the mod works for you, I would switch to that routine.
Add Thank You Quote this message in a reply
May. 22, 2010, 02:28 AM (This post was last modified: May. 22, 2010 02:45 AM by Graycode.)
Post: #29
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 22, 2010 02:19 AM)whenever Wrote:  Graycode, what's the difference between reading character by character and reading a bunch from a socket?

Sorry, I removed that post when I saw you'd responded to JJoe. Did that to avoid interfering with your project. I'll re-upload it if you want, or you can (it's Your ideas & project).

The character-by-character was to account for commenting out the setblocking(0), while also being able to accommodate either a short or a long URL.

If setblocking(0) does what I think it does then it's unlikely to get the browser's headers with the same packets that completed the SSL handshake. Without waiting (blocking) for data the read() wouldn't have anything in the queue, perhaps leading to that exception. Sometimes a bit of the data might be there, sometimes not.

So I figured that using socket blocking was better, and then grabbing the URL char-by-char seemed a reasonable solution to making sure the minimum amount of expected data had arrived before proceeding.
Add Thank You Quote this message in a reply
May. 22, 2010, 04:11 AM
Post: #30
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 22, 2010 02:28 AM)Graycode Wrote:  So I figured that using socket blocking was better, and then grabbing the URL char-by-char seemed a reasonable solution to making sure the minimum amount of expected data had arrived before proceeding.

Thanks. Here is the 0.1b version using the char-by-char routine, though I am wondering if it would solve JJoe's issue.


Attached File(s)
.zip  ProxHTTPSProxy 0.1b.zip (Size: 1.36 KB / Downloads: 756)
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: