Post Reply 
ProxHTTPSProxyMII: Reloaded
Nov. 20, 2014, 08:47 AM
Post: #16
RE: ProxHTTPSProxyMII
(Nov. 20, 2014 07:51 AM)laighleas Wrote:  Ah! So you can get the ProxHTTPSProxy/Proxomitron complex to work with an upstream anonymising proxy?

I think the feature you mentioned is already implemented in ProxHTTPSProxy, you can see the config file, they have a few line about fake ip, just use ProxHTTPSProxy as parent proxy and fake ip with your proxy.
Add Thank You Quote this message in a reply
Nov. 20, 2014, 09:38 AM
Post: #17
RE: ProxHTTPSProxyMII
Ok, good program. :-) A few wrinkles to iron out.

Firstly, the readme says:

"Add to Proxomitron's "Bypass URLs that match this expression" field if it is empty"

`$OHDR(Tagged:Proxomitron FrontProxy/*)$SETPROXY(127.0.0.1:8081)(^)`

I'm not using Sidki's filters - I never understood them - and I don't have a "Bypass URLs that match this expression" section. So, header or webpage filter?

Next a couple of error messages:

For Browserspy.dk

Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:600).

For this site:

Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - hostname 'prxbx.com' doesn't match either of '*.hostwhitelabel.com', 'hostwhitelabel.com'.

Finally: "I think the feature you mentioned is already implemented in ProxHTTPSProxy, you can see the config file, they have a few line about fake ip, just use ProxHTTPSProxy as parent proxy and fake ip with your proxy."

I want to chain it with something like JonDo. No point in having an anonymising proxy if unfiltered https can get through and reveal details. :-)
Add Thank You Quote this message in a reply
Nov. 21, 2014, 12:06 AM
Post: #18
RE: ProxHTTPSProxyMII
Well, I seem to have broken it, and I can't for the life of me work out what I'm doing wrong. I'm getting a lot of sites with error messages like the one from http://www.cvedetails.com/vulnerability-...efox.html:


Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600).


ProxHTTPSProxy's own message window says that the connection timed out [SSL error]

Other places e.g http://www.bristolbotanicals.co.uk/ I'm getting:


The page isn't redirecting properly

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.


The message window just shows an unending column of GET commands.

And for http://www.proxomitron.info/45/help/Matc...mands.html


Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - hostname 'www.proxomitron.info' doesn't match either of '*.powweb.com', 'powweb.com'.


That's pretty much the majority of web pages I've tried. I've got Proxomitron completely bypassed, so it's nothing to do with the web filters. The sites work with HTTPS Everywhere.

I haven't done anything to config.ini. It's showing:

ProxAddr = http://localhost:8080
FrontPort = 8079
RearPort = 8081

Firefox is set for HTTP 8080 and HTTPS 8079. Proxomitron is set to use 8081. HTTPS Everywhere is disabled. The certificate has been imported (BTW do I do anything with private.key?). The minimum SSL that Firefox accepts is TLS 1.0.

I did have it working, though without the filter. I did throw one together, though I've no idea if I got it right. It doesn't appear in the main window when I select header filters, so presumably there's something wrong with it:

In = FALSE
Out = TRUE
Key = "Redirect to HTTPS proxy"
Match = "$OHDR(Tagged:Proxomitron FrontProxy/*)$SETPROXY(127.0.0.1:8081)(^)"

Anyway, I'm damned if I know what's gone wrong, it is midnight, and I've spent the last two hours unsuccessfully trying to get it to work again.

Help! :-(
Add Thank You Quote this message in a reply
Nov. 21, 2014, 12:12 AM
Post: #19
RE: ProxHTTPSProxyMII
Oh, and OpenSSL is installed and in the correct directory.
Add Thank You Quote this message in a reply
Nov. 21, 2014, 12:54 AM
Post: #20
RE: ProxHTTPSProxyMII
No idea what happened but forty minutes later and I'm not getting the error messages. :-/

It's not filtering https though. Probably the fault of the filter. Too tired to think any more. :-(
Add Thank You Quote this message in a reply
Nov. 21, 2014, 08:59 AM
Post: #21
RE: ProxHTTPSProxyMII
Still no luck.

The error messages appear when the Use Proxy checkbox is checked. At Bristol Botanicals I get:

+++GET 876+++
Using Proxy - 127.0.0.1:8081
GET http://www.bristolbotanicals.co.uk/baske...%2Fpr-2390 HTTP/1.1
Host: http://www.bristolbotanicals.co.uk
User-Agent: Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.2; SLCC1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: deflate, gzip
DNT: 1
Cache-Control: no-store
Connection: close

+++RESP 876+++
HTTP/1.1 302 Moved Temporarily
Date: Fri, 21 Nov 2014 08:54:47 GMT
Server: Apache
X-Powered-By: PHP/5.4.32
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: BristolBotanicalsSessionID=b12fa454f2a6ecc7fa6e6da7350fd0ce; path=/
Location: http://www.bristolbotanicals.co.uk/baske...%2Fpr-2390
Content-Type: text/html
Content-Length: 0
Connection: close
+++CLOSE 876+++

Which results in a lot of GET commands in ProxHTTPSProxy's window and an error message in Firefox. :-(

The page isn't redirecting properly

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

Filtering is going on - I can see that - but I keep getting error messages unless I uncheck the Use Proxy box.

Don't have a clue how to solve this. :-(

OTOH, using a direct connection and HTTPS Anywhere and hey presto! I get the page. :-(

Anyone got a solution?
Add Thank You Quote this message in a reply
Nov. 21, 2014, 09:50 AM
Post: #22
RE: ProxHTTPSProxyMII
Browserspy produces the following logs:

+++GET 881+++
Using Proxy - 127.0.0.1:8081
GET http://browserspy.dk/ HTTP/1.1
Host: browserspy.dk
User-Agent: Mozilla/5.0 (Windows; U; BeOS; en-US; rv:1.9.0.7) Gecko/2009021910
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: deflate, gzip
DNT: 1
If-Modified-Since: Fri, 14 Nov 2014 09:27:37 GMT
Cache-Control: no-store
Connection: close

+++RESP 881+++
HTTP/1.1 417 Exception <class 'urllib3.exceptions.SSLError'>
Server: Proxomitron RearProxy/1.1 Python/3.4.2
Date: Fri, 21 Nov 2014 09:28:04 GMT
Content-Type: text/html;charset=utf-8
Connection: close
Content-Length: 523
<start> 881: Pop-up windows: Kill
<start> 881: Window: Stop status bar scrollers
Match 881: Kill alert/confirm boxes
Match 881: Foreign content-type filter
<end> 881: Check link targets 2
<end> 881: Disable status bar manipulation: Links
<end> 881: Disable status bar manipulation: Scripts
+++CLOSE 881+++

And error message in Firefox:

Error response
Error code: 417
Message: Exception <class 'urllib3.exceptions.SSLError'>.
Error code explanation: 417 - [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:600).

ProxHTTPSProxy says much the same thing in its window.

TUOPF gets the following server/client dialogue:

+++GET 892+++
Using Proxy - 127.0.0.1:8081
GET http://prxbx.com/forums/ HTTP/1.1
Host: prxbx.com
User-Agent: Opera/9.80 (Windows NT 6.1; U; zh-cn) Presto/2.6.37 Version/11.00
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: deflate, gzip
DNT: 1
Connection: close

+++RESP 892+++
HTTP/1.1 417 Exception <class 'urllib3.exceptions.SSLError'>
Server: Proxomitron RearProxy/1.1 Python/3.4.2
Date: Fri, 21 Nov 2014 09:30:52 GMT
Content-Type: text/html;charset=utf-8
Connection: close
Content-Length: 551
<start> 892: Pop-up windows: Kill
<start> 892: Window: Stop status bar scrollers
Match 892: Kill alert/confirm boxes
Match 892: Foreign content-type filter
<end> 892: Check link targets 2
<end> 892: Disable status bar manipulation: Links
<end> 892: Disable status bar manipulation: Scripts
+++CLOSE 892+++
BlockList 893: in UserAgents2, line 182

Error response
Error code: 417
Message: Exception <class 'urllib3.exceptions.SSLError'>.
Error code explanation: 417 - hostname 'prxbx.com' doesn't match either of '*.hostwhitelabel.com', 'hostwhitelabel.com'.

ProxHTTPSProxy’s own window points to this section on this page:

InsecureRequestWarning
New in version 1.9.
Unverified HTTPS requests will trigger a warning:
urllib3/connectionpool.py:736: InsecureRequestWarning: Unverified HTTPS
request is being made. Adding certificate verification is strongly advised.
See: https://urllib3.readthedocs.org/en/latest/security.html
This would be a great time to enable HTTPS verification: Using Certifi with urllib3.
If you know what you’re doing and would like to disable this and other warnings, you can use disable_warnings():
import urllib3
urllib3.disable_warnings()
Making unverified HTTPS requests is strongly discouraged.

https://urllib3.readthedocs.org/en/latest/security.html

But that means buggering around with Python, which is beyond me. :-(

I've hit the limits of my poor level of knowl;edge. Solving the SSL errors are way beyond me. However is it possible to write a filter that deals with ‘HTTP/1.1 302 Moved Temporarily’? And any way to get rid of ‘Server: Proxomitron RearProxy/1.1 Python/3.4.2’ that turns up in the server responses? Presumably the proxy is handing that to the server.
Add Thank You Quote this message in a reply
Nov. 21, 2014, 10:13 AM
Post: #23
RE: ProxHTTPSProxyMII
BTW sorted passing it through to Tor or JonDo. This goes to Privoxy:

[PROXY https://127.0.0.1:8118]
*

Which gives the opportunity of removing ‘Server: Proxomitron RearProxy/1.1 Python/3.4.2’
Add Thank You Quote this message in a reply
Nov. 21, 2014, 03:18 PM
Post: #24
RE: ProxHTTPSProxyMII
Sorry I was called away. I'll answer laighleas.
Add Thank You Quote this message in a reply
Nov. 21, 2014, 03:27 PM
Post: #25
RE: ProxHTTPSProxyMII
S'OK JJoe. Giving up on it for the moment, leastways till you can help. Gone as far as I can go, but can't get it to work, which is unusual for me. :-( Got Connection Timed Out errors, Connection Refused errors, 302s, bad gateways, SSL errors etc etc in ProxHTTPSProxy's window. :-(

I seem to have got round the SSL errors by telling ProxHTTPProxy to ignore verification for every site. Maybe. It's difficult to tell. :-) Not ideal. However, given that I want Proxomitron to filter https, there doesn't seem to be an alternative. :-/ I'm not going to do banking with it.
Add Thank You Quote this message in a reply
Nov. 21, 2014, 04:58 PM
Post: #26
RE: ProxHTTPSProxyMII
(Nov. 20, 2014 09:38 AM)laighleas Wrote:  I don't have a "Bypass URLs that match this expression" section.

Yes you do, Config>>Startup.

.png  prefdia.png (Size: 125.14 KB / Downloads: 774)

(Nov. 21, 2014 12:06 AM)laighleas Wrote:  Well, I seem to have broken it, and I can't for the life of me work out what I'm doing wrong. I'm getting a lot of sites with error messages like the one from http://www.cvedetails.com/vulnerability-...efox.html:

https requests are being sent to http sites...

(Nov. 21, 2014 12:06 AM)laighleas Wrote:  I haven't done anything to config.ini. It's showing:

ProxAddr = http://localhost:8080
FrontPort = 8079
RearPort = 8081

Firefox is set for HTTP 8080 and HTTPS 8079. Proxomitron is set to use 8081.


Having Proxomitron set to use 8081 will send all requests through the ProxHTTPSProxyMII rear server. ProxHTTPSProxyMII is for https only.

The "Bypass URLs that match this expression" entry's purpose is to send only https requests to the ProxHTTPSProxyMII rear server. Proxomitron's "Use Remote Proxy" switch should not be enabled.

So start fresh,
Add the ProxHTTPSProxy rear server to the Proxomitron's list of external proxies
Add the "Bypass URLs that match this expression" entry
(Do not enable Proxomitron's "Use Remote Proxy" switch)
and try it again.

HTH
Add Thank You Quote this message in a reply
Nov. 21, 2014, 05:36 PM
Post: #27
RE: ProxHTTPSProxyMII
Take it at Idiot's Guide level - in Firefox, what port for HTTP, what port for HTTPS, and so on. I am sometimes a Bear of Very Little Brain. :-)
Add Thank You Quote this message in a reply
Nov. 21, 2014, 06:33 PM
Post: #28
RE: ProxHTTPSProxyMII
(Nov. 21, 2014 05:36 PM)laighleas Wrote:  in Firefox, what port for HTTP, what port for HTTPS, and so on.

Assuming

ProxHTTPSProxyMII config.ini is
ProxAddr = http://localhost:8080
FrontPort = 8079
RearPort = 8081

The Proxomitron should be listening on Port 8080, Config>>HTTP Proxomitron's proxy port number. This is the default.

Browser HTTP should be sent to the Proxomitron at 127.0.0.1 port 8080.
Browser HTTPS should be sent to the ProxHTTPSProxyMII front server at 127.0.0.1 port 8079.

So HTTP is sent to the Proxomitron and then forwarded to the remote server as usual.

HTTPS, however, is first sent to the ProxHTTPSProxyMII front server at 8079.
The front server handles the secure connection with the browser, "tags" the request as https, and forwards the request to the Proxomitron at 8080.
The Proxomitron forwards the "tagged" http requests to the ProxHTTPSProxyMII rear server at 8081.
The rear server handles the secure connection with the remote https server.

The "Bypass URLs that match this expression" entry tells the Proxomitron to forward "tagged" requests to the rear server.

HTH
Add Thank You Quote this message in a reply
Nov. 21, 2014, 09:17 PM (This post was last modified: Nov. 21, 2014 09:18 PM by laighleas.)
Post: #29
RE: ProxHTTPSProxyMII
Yup, got that. The original instructions were a bit unclear, and I had it set up wrong in Firefox, but I went back, thought about it from first principles, and set it to the values you've given above.

I put in the "Bypass URLs that match this expression and can see from the log window that it is working:

$OHDR(Tagged:Proxomitron FrontProxy/*)$SETPROXY(127.0.0.1:8081)(^)

I also put in the line for things that do *not* match: $OHDR(Tagged:Proxomitron FrontProxy/*)$SETPROXY(127.0.0.1:8081)(^)|

Both are marked Out.

That correct so far?

I'll have to pick this up tomorrow, since it is small child's bedtime.

Oh, and "Use remote proxy" is set, and the proxy is set to port 8081.
Add Thank You Quote this message in a reply
Nov. 21, 2014, 09:21 PM
Post: #30
RE: ProxHTTPSProxyMII
Even so, http://www.bristolbotanicals.co.uk/baske...%2Fpr-2390

The page isn't redirecting properly

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

302 error :-(

Browserspy.dk

Error response

Error code: 417

Message: Exception <class 'urllib3.exceptions.SSLError'>.

Error code explanation: 417 - [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:600).

:-(
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: