Proxomitron Reborn

[zoltan]

Maybe I misunderstood. I only changed the HTTPS port number to 8443 under the HTTPS tab in Proxomitron. I didn't change it in Firefox's Network settings. It's still 8080 there.

The only other difference I can see is that I'm on Win 7 and am using the installed version of FF 115esr.

[JJoe]

(Feb. 02, 2025 07:03 PM)zoltan Wrote:  I only changed the HTTPS port number to 8443 under the HTTPS tab in Proxomitron. I didn't change it in Firefox's Network settings. It's still 8080 there.

Correct.

(Feb. 02, 2025 07:03 PM)zoltan Wrote:  The only other difference I can see is that I'm on Win 7 and am using the installed version of FF 115esr.

If you want to try it, I downloaded the FirefoxPortableESR_115.11.0_English.paf.exe from https://sourceforge.net/projects/portableapps/files/Mozilla%20Firefox,%20Portable%20Ed./Mozilla%20Firefox%20ESR,%20Portable%20Edition%20115.11.0/

[JJoe]

(Feb. 02, 2025 07:03 PM)zoltan Wrote:  The only other difference I can see is that I'm on Win 7 and am using the installed version of FF 115esr.

Your proxo.exe is dated 4/23/2024. Mine is 4/22/2024. Are you using the ProxomitronReborn_4701R.zip exe?

For certificate generation try: ST = FL; C = US ; Signature algorithm = RSA-SHA256
or
https://www.prxbx.com/forums/showthread....5#pid20945

[amy]

Also select 2048 for RSA Key Size, as 1024 is not considered secure enough these days. (This is only securing the connection between the proxy and the browser, so it's pretty much irrelevant if they're both running on the same machine.)

[zoltan]

Note: You can skip to the Edit at bottom as the problem with no filtering was my mistake. Short version: it seems to be working now !! --- except for google.

••••

I tried FF 115esr portable. At first, using the same proxert_certonly.pem that I had previously generated, I got the same errors: "site.com uses an invalid security certificate. The certificate does not come from a trusted source. Error code: Mozilla PKIX error CA Cert used as end entity."

But after regenerating the certificate according to the above instructions, I get no errors, but everything is completely unfiltered. It's as if Proxomitron is off. And yes, I've double checked the browser's network settings to make sure manual proxy is checked and both are set to 8080. I also tried importing the linked ProxomitronSSLFilteringRootCA.crt but got the same result of no errors/no filtering.

To be certain, I downloaded Proxo 4701R again, and it shows 4/23/24 12:49am. Could that just be Windows displaying time an hour later because it was saved during standard time vs DST? The file is from the end of amy's original post in this thread.

I wondered if it had something to do with the portable Firefox configuration, which I'm not as familiar with, but after switching over to ProxHTTPSProxy, the same browser is filtering normally.

•••••

Edit:
Well, that was embarrassing..... I started remembering when I first tried Reborn 6 years ago and was having similar problems but had forgotten most of it until I went back and just now reread my first posts in this thread. At the time, "UseSSLeay..." was unchecked. Somehow last night during the last certificate regeneration it got unchecked again. How, I don't know, but now that it's checked, the first few sites I've tried (except for google which won't load) are filtering again, this time without errors, so the steps above with certificate generation seem to have worked. To assess it properly I'm going to have to go back to my installed version of Firefox where about:config and extensions are customized as I use them.

[zoltan]

After some testing on my installed FF 115esr, most things are loading normally without cert errors. I'm very pleased to get Reborn working to this extent after all this time. But there are some issues:

Something is apparently wrong with $JUMP. For example, I have a good many Exceptions-U entries like this one:

Code:

www.wikipedia.org/   $JUMP(http://en.wikipedia.org/wiki/Special:Search)

Entering http://www.wikipedia.org attempted the jump and produced "SSL ERROR BAD CERT DOMAIN"
When I went directly to the Special:Search page by bookmark it loaded fine.
Strangely, after changing the jump-to address to https and reloading lists it worked. After changing it back to http, it STILL works. How is that even possible if it fails originally and you reproduce those original conditions?

Now that I can't reproduce the failure on that one, I tried more. For example,

Code:

www.wunderground.com    $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

fails with a "page isn't redirecting properly" error. Proxomitron log looks like this. If you scrolll down that log, there are 21 repeated attempts to jump. Most pages seem to behave this way instead of the Bad Cert Domain error I got when Wikipedia was jumped.

Using $JUMP to go to local.ptron images or pages seems to work normally without problems.

$JUMP issues may also be the culprit in why google isn't working. Long ago I added a rather complex google redirect that was developed in this thread with help from JJoe.

Code:

www.google.com/ (webhp\?hl=en$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search)
                 |firefox$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search)
                 |imghp$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_image_search?hl=en)
                 |?$SET(0=f_ua_ff.a_js.)
                 |$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search))

It's been working great for 13 years to direct the main search page URL to the Advanced page, but now with Reborn it's preventing access to google whether it's an attempted jump or whether you enter the address for Advanced Search directly. This jump and others work without issue through ProxHTTPSProxy. The original conditions that required the browser fakes don't really apply anymore, so I'd be happy if it was just a simple jump to advanced search that worked.

I've also found on some sites (reddit or wunderground for example) that if a page is loaded when bypassed, the bypass can't be undone by unchecking it in the menu. You have to actually restart Reborn in order to reload the page with filtering. The effect doesn't seem to transfer to another site if you uncheck bypass before loading it. So far, about half the sites I've tried have this problem. In this and the other instances above, I'm always clearing cache before page loads.

[JJoe]

(Feb. 04, 2025 02:01 AM)zoltan Wrote:  For example,

Code:

www.wunderground.com    $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

fails with a "page isn't redirecting properly" error. Proxomitron log looks like this. If you scrolll down that log, there are 21 repeated attempts to jump. Most pages seem to behave this way instead of the Bad Cert Domain error I got when Wikipedia was jumped.

This is as expected.
With $JUMP() the Proxomitron instructs the browser to make a new request to the target URL. The new request is filtered by the Proxomitron. In this case

Code:

www.wunderground.com    $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

always matches the request for https://www.wunderground.com/forecast/us/mn/minneapolis. Scott added code (21 repeated attempts) to prevent an endless or infinite loop.
You need to add some code so that the target URL is not matched.

(Feb. 04, 2025 02:01 AM)zoltan Wrote:  I've also found on some sites (reddit or wunderground for example) that if a page is loaded when bypassed, the bypass can't be undone by unchecking it in the menu. You have to actually restart Reborn in order to reload the page with filtering. The effect doesn't seem to transfer to another site if you uncheck bypass before loading it. So far, about half the sites I've tried have this problem. In this and the other instances above, I'm always clearing cache before page loads.

While the Proxomitron was in bypass the browser and the site have agreed to use something that the Proxomiton does not understand, probably encoding or the version of HTTP.
I haven't seen this, since I disabled http2.

[JJoe]

I think this

(Feb. 04, 2025 02:01 AM)zoltan Wrote:  ...
Long ago I added a rather complex google redirect that was developed in this thread with help from JJoe.

Code:

www.google.com/ (webhp\?hl=en$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search)
                 |firefox$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search)
                 |imghp$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_image_search?hl=en)
                 |?$SET(0=f_ua_ff.a_js.)
                 |$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search))

It's been working great for 13 years to direct the main search page URL to the Advanced page, but now with Reborn it's preventing access to google whether it's an attempted jump or whether you enter the address for Advanced Search directly. This jump and others work without issue through ProxHTTPSProxy. The original conditions that required the browser fakes don't really apply anymore, so I'd be happy if it was just a simple jump to advanced search that worked.
...

is due to

(Jun. 16, 2024 04:48 AM)amy Wrote:  

(May. 18, 2024 03:40 AM)JJoe Wrote:  $RDIR and $JUMP appear to be broken but not always.

$RDIR and $JUMP interact badly with HTTPS filtering because in that case there are actually two requests, the first with a CONNECT method to open the encrypted tunnel, and then the "real" request is sent inside that. This also means outbound header filters are applied twice on each request, one for the outer and another for the inner. That first one with the CONNECT has no path, which explains the bug. I suspect this was something Scott didn't realise, since he didn't see HTTPS filtering as anything but an experimental feature. Thanks for the detailed test cases. This will be fixed in the next release.

This also makes creating an expression to $JUMP from www.wunderground.com to https://www.wunderground.com/forecast/us/mn/minneapolis difficult tricky.

Edit to add a thought:
In the Proxomitron's log window, the Host header for CONNECT ends with ":443". The "real" request's Host header does not. $OHDR(Host:\1) does reflect this.
We may be able to use this to our advantage...

Edit to propose:

Code:

www.google.com/ $OHDR(Host:www.google.com(^?))(webhp\?hl=en$JUMP(https://www.google.com/advanced_search)
                 |firefox$JUMP(https://www.google.com/advanced_search)
                 |imghp$JUMP(https://www.google.com/advanced_image_search?hl=en)
                 |(^?)$JUMP(https://www.google.com/advanced_search))

which appears to work for me.

Edit: strike though difficult and add tricky.

[zoltan]

Thanks! That does work for the google jump. The header request connect concept strays into territory I don't fully understand, but I tried using your new filter as a model for the simpler one:

Code:

www.wunderground.com/ $OHDR(Host:www.wunderground.com(^?))   $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

but it still results in "...page isn't redirecting properly"
I don't understand the part about the error being "expected" and needing to add code "so that the target URL is not matched." The $JUMP, as previously constructed, is just like many that I've been using for a long time successfully, so something different is happening, but I guess that's what you meant by broken and what amy meant about being fixed in the next release. In the meantime, I'll be glad to use whatever "tricks" work.

The stuck bypass issue has become more of a problem than I thought at first because with troubleshooting filters I'm having to frequently close and reopen proxo to get filtering back. I can go back to ProxHTTPSProxy if necessary, but I'm just reporting what I experience if Reborn is still in development. It's hard to know if something is just a quirk of my setup or whether others are experiencing these things too.

[JJoe]

(Feb. 06, 2025 07:48 AM)zoltan Wrote:  ...
I tried using your new filter as a model for the simpler one:

Code:

www.wunderground.com/ $OHDR(Host:www.wunderground.com(^?))   $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

but it still results in "...page isn't redirecting properly"
I don't understand
...

Does this help?

Code:

www.wunderground.com/(^?) $OHDR(Host:www.wunderground.com(^?))   $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

This expression only matches "www.wunderground.com/"
So there won't be a JUMP for www.wunderground.com/morehere.

(Feb. 06, 2025 07:48 AM)zoltan Wrote:  ...
The $JUMP, as previously constructed, is just like many that I've been using for a long time successfully, so something different is happening
...

It could be that the reason it worked for you past was that you JUMPed from http to https.
www.wunderground.com/ would not match www.wunderground.com:443/

Due to the popularity of https, amy has made some changes.

Going forward and with your format, I'd expect a loop for $JUMP and no loop for $RDIR.

In the old google expression,

Code:

?$SET(0=f_ua_ff.a_js.)

prevented the loop by matching www.google.com/morehere before

Code:

$SET(0=f_ua_ff.)$JUMP(https://www.google.com/advanced_search)

could.

$JUMP sends a command to the browser to request an address. The Proxomitron will filter this new request.
$RDIR tells the Proxomitron to request an address itself. So, no browser loop. The Proxomitron won't tell the browser about the redirection.

[zoltan]

That's working perfectly. I used the same format with other sites I redirect and they're working too.

I went back to ProxHTTPSProxy and tested with just https and the redirect worked with:

Code:

www.wunderground.com/(^?)    $JUMP(https://www.wunderground.com/forecast/us/mn/minneapolis)

With Reborn, it does require the addition of $OHDR(Host:www.wunderground.com(^?)) That's fine though, as long as I know what to do.

Appreciate the $RDIR vs $JUMP details. I didn't know the process was so different. I'll need to rethink my use of these.

[zoltan]

Coming from years of using ProxHTTPSProxy, I'm uncertain about a few things and how Reborn may differ. With the former, there is a config.ini file that allows you to add URLs to an "SSL Pass-Thru" which seemed to be the equivalent of choosing No Proxy in the browser's settings. This was important for some sites (or their .css or .js hosts) that absolutely would fail under all filtering circumstances. Putting those URLs in the Proxomitron's Bypass list was not helpful. It would stop filtering, but not SSL errors.

With Reborn there's obviously no such config.ini, so I'm wondering if the Bypass list is now the equivalent of what I was doing with the SSL-Pass-Thru in config.ini.

The ProxHTTPSProxy's comand prompt window also allowed an instant view of which specific URLs were triggering errors, which was important for knowing which URLs to add to the pass-thru. Now I'm feeling kind of in the dark about what's going on underneath Reborn when something's not working.

Which brings me to google maps not loading. At first I thought it could be the $JUMP discussed above, but after commenting it out, maps still fails. Since it's failing with the same config that loads under ProxHTTPSProxy, I'm at a loss for how to start troubleshooting. Ordinarily, I'd first look at the prompt window. So far, all I know is that it's solved by unchecking "Web Page Filters" but I'm wondering if that may be producing a broader effect than it was by unchecking the same with ProxHTTPSProxy's setup. If it's one of those situations where one of the site's many dependent files are being blocked with an SSL error that's not apparent with a browser notification, how do you solve it? Not so much for this instance but any future ones as well.

I created a test config where I stripped out all filters below "Patterns." With that, google maps still fails, so it's definitely not the same as unchecking Web Filters. I didn't think my filters were actually a problem anyway since maps works with the same filters under ProxHTTPSProxy.

Google Images is also failing to an extent. It loads the thumbnails, but clicking them produces no response.