Best settings to filter https ?

[leecovuk]

Hello everyone, I hope you're all well.

I'm using Sidki's config from February with the June update on it.

For a while I haven't been filtering https through Prox, as I sometimes felt it/I wasn't doing it correctly/ideally. I've been attempting to stop https filtering by setting the proxy settings in Firefox and IE to only pass http through Prox, and I've also set there to never use the proxy for sites starting with https. I don't know if that is all necessary or correct.
A slight complication I have is that I still like to use JJoe's YLogin script, so I think I can't just un-check the ssl dlls in the general Prox config.
Anyway, for simplicity, let's forget that latter issue for now.

If I want to resume filtering https through Prox and be confident about it, can I please check the best way of setting it up, both in the browsers and Prox config? I loosely understand the idea of half-ssl regarding avoiding certificate security warnings, but Sidki's config seems to have 2 half-ssl filters available under Web Filters and one under Header Filters, and I've never understood which ones to select.

Many thanks,
Lee
UK

[JJoe]

(Oct. 22, 2009 01:01 PM)leecovuk Wrote:  A slight complication I have is that I still like to use JJoe's YLogin script, so I think I can't just un-check the ssl dlls in the general Prox config.

Try it. I think it will work. The dlls must be available and $SET(YLaction=http://https.)(^) active, however.

(Oct. 22, 2009 01:01 PM)leecovuk Wrote:  If I want to resume filtering https through Prox and be confident about it,

"be confident about it"
?

(Oct. 22, 2009 01:01 PM)leecovuk Wrote:  but Sidki's config seems to have 2 half-ssl filters available under Web Filters and one under Header Filters, and I've never understood which ones to select.

The Header filter enables the others.

HTH

[JJoe]

When the Proxomitron intercepts the browser's https,
the Proxomitron may modify/filter the headers seen in the Log window.

When all the SSL files are available,
the Proxomitron can create https from the browser's http with $RDIR(https://addresshere) and filter the page.

When all the SSL files are available and URL Commands are enabled,
the Proxomitron can create https from the browser's http with 'http://https..' and filter the page.

When all the SSL files are available and "Use SSLeay/OpenSSL to filter secure pages" is enabled,
the Proxomitron can intercept and filter the browser's https.
The browser should warn the user.

My browser sends https to the Proxomitron.
I have "Use SSLeay/OpenSSL to filter secure pages" enabled.
I try to hide the https from the browser, half-SSL for less warnings.
I use a direct connection when I don't want any mistakes.
Best? I can't say.

HTH

[leecovuk]

Hello JJoe, I've just seen your replies, and thanks.

I've just looked more closely and can see there is more than one ssl related header filter in Sidki's config, so to clarify then, assuming at this stage I'm not using or have merged in your YLogin filters;

The filter I was originally enquiring about / noticed was:
! |||||||||||| 2.2 Use Half-SSL 5.01.12 [jjoe] (o.2) (Out)
under Header Filters, which is off by default.
Are you saying that enabling that just achieves the same result as enabling
<*>: Half-SSL 8.03.06 (cch! multi) [sd jjoe] (d.2)
and
JS CSS: Half-SSL 7.11.02 (cch! multi) [jjoe sd] (d.2)
under Web Page filters, which are both enabled by default.

Have I got that right? If so, presumably the header filter can be ignored if the 2 web page filters are already on?

Then, regarding how to set up the browser proxy settings;
should we/are you suggesting, JJoe, we should route both http and https through localhost/127.0.0.1:[Prox's listening port]?

Then finally, if I/we use your YLogin filters, just do the config merge and we're done?
(By 'done' I mean theoretically set up for filtering all secure pages/sites, along with using your YLogin filters, with the lowest probability of getting certificate security warnings)

Thanks again,
Lee

[JJoe]

(Oct. 24, 2009 04:15 PM)leecovuk Wrote:  I've just looked more closely and can see there is more than one ssl related header filter in Sidki's config, so to clarify then, assuming at this stage I'm not using or have merged in your YLogin filters;

The filter I was originally enquiring about / noticed was:
! |||||||||||| 2.2 Use Half-SSL 5.01.12 [jjoe] (o.2) (Out)
under Header Filters, which is off by default.
Are you saying that enabling that just achieves the same result as enabling
<*>: Half-SSL 8.03.06 (cch! multi) [sd jjoe] (d.2)
and
JS CSS: Half-SSL 7.11.02 (cch! multi) [jjoe sd] (d.2)
under Web Page filters, which are both enabled by default.

No.
The set uses a variable to control all the Half-SSL related processes.
The header filter '2.2 Use Half-SSL' can create that variable.
'<*>: Half-SSL' and 'JS CSS: Half-SSL' are on by default but require the variable to work.
So, '2.2 Use Half-SSL' is a 'trigger' or 'toggle' filter that starts the Half-SSL process.

The user only enables/disables one filter instead of 4 or more.
Seemed like a user friendly feature.

(Oct. 24, 2009 04:15 PM)leecovuk Wrote:  Then, regarding how to set up the browser proxy settings;
should we/are you suggesting, JJoe, we should route both http and https through localhost/127.0.0.1:[Prox's listening port]?

I do. I route http and https through 127.0.0.1:8080.
I see warnings from the browser about the Proxomitron and site.
I see warnings and errors from the Proxomitron.

I know somebody who only filters Half-SSLed pages.
He sees warnings from the browser about the site.
He see warnings and errors from the Proxomitron.
He sees less warnings and errors and more unwanted content than I do.
He does more https than I do and has https bookmarks.

(Oct. 24, 2009 04:15 PM)leecovuk Wrote:  Then finally, if I/we use your YLogin filters, just do the config merge and we're done?
(By 'done' I mean theoretically set up for filtering all secure pages/sites, along with using your YLogin filters, with the lowest probability of getting certificate security warnings)

YLogin requires a choice, https or http://https.

Code:

# Remove one of the # from the two lines below.

#$SET(YLaction=https://)(^)
#$SET(YLaction=http://https.)(^)

#  Removing the # from the #$SET(YLaction=https://)(^) line above
#   provides a secure (https) connection between the browser and the
#   Proxomitron. The browser may complain about the Proxomitron.
#  Removing the # from the #$SET(YLaction=http://https.)(^) line above
#   provides a 'HalfSSL' connection. The browser will not see the
#   secure (https) connection and should not complain about the
#   Proxomitron.
(^$TST(YLaction=?*))$ALERT(YLaction not set!\r\nEdit line 13 or 14 of YLogin.txt)

So to filter all secure pages/sites, use YLogin, and see the least alerts and warnings:
Route both http and https through localhost/127.0.0.1:[Prox's listening port]
Enable the header filter ! |||||||||||| 2.2 Use Half-SSL 5.01.12 [jjoe] (o.2) (Out) or equivalent.
Add YLogin and edit line 14 of YLogin.txt.

I think, Sidki's Half-SSL option will hide the https connection regardless.

Again tho, no mistakes may require a direct connection with https and http.

HTH

[leecovuk]

Thanks again for that JJoe, I appreciate your effort.

Regarding YLogin, I remember looking at and setting those options; I have just looked at YLogin.txt and I am set as follows:
#$SET(YLaction=https://)(^)
$SET(YLaction=http://https.)(^)

As always with me, I will just end up confusing myself and re-treading already explained ground. I just wanted to generally check I was filtering https in the least 'annoying' way whilst doing it as effectively as possible.
For a while, for example, I recall I was wanting to use half-ssl with the understanding that filtering https through Prox using half-ssl helps avoid certificate warnings, but I was only passing http through Prox in the browser proxy settings. I forget now my reasoning for doing that, but presumably it was to try avoiding some remaining certificate warnings, and it appeared to be doing filtering of (some/all?) secure pages.

Quote:I know somebody who only filters Half-SSLed pages.

Is that what I have just described above?
Do they only route http through Prox in the browser proxy settings?
Or do you mean they only use Prox to filter secure pages and not any http? (presumably not)

Finally, should I not want to filter a certain https url, what syntax would you use in the bypass list or IncludeExclude-U.ptxt? I know IncludeExclude-U.ptxt has the following:

don't use half-SSL (if default) $SET(0=i_ssl_h:0.)

but it is the url matching expressions which consistently elude me.
If you like, let's use an example that would match both
https://subdomain.domain.com
and
https://www.subdomain.domain.com

but not

http://subdomain.domain.com
and
http://www.subdomain.domain.com

Thanks again,
Lee

[JJoe]

(Oct. 25, 2009 12:51 AM)leecovuk Wrote:  Regarding YLogin, I remember looking at and setting those options; I have just looked at YLogin.txt and I am set as follows:
#$SET(YLaction=https://)(^)
$SET(YLaction=http://https.)(^)

As always with me, I will just end up confusing myself and re-treading already explained ground. I just wanted to generally check I was filtering https in the least 'annoying' way whilst doing it as effectively as possible.

So you have been hiding the secure connection from the browser (aka using Half-SSL) to avoid seeing the 'who is Proxomitron' warning.

(Oct. 25, 2009 12:51 AM)leecovuk Wrote:  For a while, for example, I recall I was wanting to use half-ssl with the understanding that filtering https through Prox using half-ssl helps avoid certificate warnings, but I was only passing http through Prox in the browser proxy settings. I forget now my reasoning for doing that, but presumably it was to try avoiding some remaining certificate warnings, and it appeared to be doing filtering of (some/all?) secure pages.

Quote:I know somebody who only filters Half-SSLed pages.

Is that what I have just described above?

Close.
Your browser's https saw no filtering. There were no warnings about the Proxomitron.
I believe he routes http and https through the Proxomitron but disables "Use SSLeay/OpenSSL to filter secure pages".
So, his browser's https sees minimal header filtering. The files are not modified. There are no warnings about the Proxomitron.

Either method:
Https addresses found in headers or web pages and converted by Half-SSL routines may be filtered.
Half-SSL addresses requested by the browser may be filtered.
Should generate less certificate warnings and errors because the Proxomitron is older. Modern browsers should make fewer mistakes but they still don't filter.

Oct 29, 2012 It took awhile to get back to this. Lee was asking how to disable filtering. For some reason I showed how to enable.

(Oct. 25, 2009 12:51 AM)leecovuk Wrote:  Finally, should I not want to filter a certain https url, what syntax would you use in the bypass list or IncludeExclude-U.ptxt? I know IncludeExclude-U.ptxt has the following:

don't use half-SSL (if default) $SET(0=i_ssl_h:0.)

but it is the url matching expressions which consistently elude me.
If you like, let's use an example that would match both
https://subdomain.domain.com
and
https://www.subdomain.domain.com

but not

http://subdomain.domain.com
and
http://www.subdomain.domain.com

Oct 29, 2012 Why and how $SET(0=i_ssl_h:1.) and $SET(0=i_ssl_h:2.) enable filtering follows

I think that should be
$SET(0=i_ssl_h:1.)

Code:

[HTTP headers]
In = FALSE
Out = TRUE
Key = "! |||||||||||| 2.2 Use Half-SSL     5.01.12 [jjoe] (o.2) (Out)"
URL = "$SET(keyword=$TST(keyword=(^*.i_ssl_h:)\1)\1i_ssl_h:1.)"

or

$SET(0=i_ssl_h:2.)

Code:

[Patterns]
Name = "<*>: Half-SSL     8.03.06 (cch! multi) [sd jjoe] (d.2)"
Active = TRUE
Multi = TRUE
URL = "$TYPE(htm)$TST(keyword=*.i_ssl_h:[12].*)"
Bounds = "$NEST(<[abdefhilmostu],*https://*,>)"
Limit = 2048
Match = "(^$TST(comment=1)|$TST(tNoscript=1))("
        ""
        "(*\s(href|src|action|background|style|content|value|on[a-z]+)=)\#"
        "$AVQ("
        "(\\+"+ https://&\#s://$SET(#=://https-px-.)\#)"
        "|(\0https://(^$TST(\0=\\+"+ (http:/|/|..|)/*))&&\#s://$SET(#=://https-px-.))+{1,*}\#"
        ")"
        ""
        ")+{1,*}\#"
Replace = "\@"

Proxomitron adds the port for https.
So, I'd try something like:

(www.|)subdomain.domain.com:443 $SET(0=i_ssl_h:1.)

Test at https://addons.mozilla.org/en-US/firefox/
with
(www.|)addons.mozilla.org:443 $SET(0=i_ssl_h:1.)
seems to work.

HTH

Edit: Remove http:// that forum software adds to www; Try to clarify my mistakes to salvage thread.

[leecovuk]

Thanks for all that JJoe,
I do appreciate it.
Some or most of it has been covered before on the forums, some of which even in answer to me, but I was struggling to dig out the relevant topics to refer to them.

Lee

[leecovuk]

Hello again JJoe, a final thought on this has occured to me;

Were you describing above how to disable half-ssl filtering on an address, with the aim that then it would become 'full' ssl filtering?
I ask because Sidki suggested in this post:

http://prxbx.com/forums/showthread.php?t...5#pid12415

how to disable ssl filtering on a url by adding the following in the bypass list:

Match all secure pages on login.live.com: login.live.com:
Match all secure pages on the entire live.com domain: ([^/]++.|)live.com:

ie that presumably disables all ssl filtering on that address/site rather than just disabling half-ssl filtering using IncludeExclude-U.ptxt. I would imagine this is what I would need to do if I had trouble with a ssl site, rather than changing the ssl filtering method. I appreciate however that my query to you probably came across as specifically how to disable half-ssl filtering on an address/site rather than all ssl filtering on that address/site.

I'm also inclined to ask you if your reply meant that the line I quoted from IncludeExclude-U.ptxt is in fact an error:

don't use half-SSL (if default) $SET(0=i_ssl_h:0.)

but, if it's correct, I suspect I wouldn't understand the nuances of its usage anyway, compared to your description of how to use:
$SET(0=i_ssl_h:1.)
or
$SET(0=i_ssl_h:2.)

If you want however to reply on this, others reading it may find it useful.

Lee

[JJoe]

(Oct. 25, 2009 07:59 PM)leecovuk Wrote:  I appreciate however that my query to you probably came across as specifically how to disable half-ssl filtering on an address/site rather than all ssl filtering on that address/site.

I missed a question! drats...
Have to answer later.

(Oct. 25, 2009 07:59 PM)leecovuk Wrote:  I'm also inclined to ask you if your reply meant that the line I quoted from IncludeExclude-U.ptxt is in fact an error:

I think so.
I'll create a topic for it later, if needed.
Oct 29, 2012 No topic was created because
"don't use half-SSL (if default) $SET(0=i_ssl_h:0.)"
was and is correct.

Have fun

Edit: update thread.

[JJoe]

(Oct. 25, 2009 10:39 PM)JJoe Wrote:  I missed a question! drats...
Have to answer later.

The question appears to be,
'I am using sidki's set. Half-SSL is enabled. I don't want this secure site filtered. What can I add to Bypass?"

The Proxomitron can show us what an expression in Bypass List.txt needs to match to bypass a particular address.
After you add and save

Code:

\1&$LOG(R\1)(^)

to Bypass List.txt
open the Proxomitron's Log window
and load the address.

The set's Half-SSL routines change
https://addons.mozilla.org/
to
http://https-px-.addons.mozilla.org/

For
https://addons.mozilla.org/
I see
addons.mozilla.org:443/

For
http://https-px-.addons.mozilla.org/
I see
https-px-.addons.mozilla.org/

I need to bypass
addons.mozilla.org:443/
and do something about
https-px-.addons.mozilla.org/

So I'll try adding

Code:

addons.mozilla.org:443/
https-px-.addons.mozilla.org/&$JUMP(https://addons.mozilla.org/)

to Bypass List.txt

Seems to work.
I don't see any filtering.
Can you confirm?

HTH

Please note.
*Any* match in Bypass List.txt causes the Address calling the List to be bypassed!
\1&$LOG(R\1)(^) doesn't match.
\1&$LOG(R\1) does match.
Be careful in there.

[leecovuk]

hello JJoe, I've just noticed your reply.

Yes, all that makes sense, thanks for that.

Lee