Complex pages exceed log window buffer?

[Aljan Darkmoon]

Naoko 4.5-J, JD5000's last "industrial strength" filter set, Firefox 3.6.10, Windows XP Pro SP3, 2.8 GHz Pentium 4 with 1 GB RAM.

Hi,

I am attempting to log the loading of a video display page that links from http://video.pbs.org/ (any page will do-- http://video.pbs.org/video/1312522241 for example) so that I may be able to discover how to adjust my blockfiles so that these pages may be viewed properly without bypassing Prox entirely. In order to make sure that I am logging everything necessary to display the page, I am emptying Firefox's cache just prior to loading the page that I want to log.

Though I have the log window set to capture only headers and filter matches (default settings), the page seems to require more requests and responses than will fit in the log window buffer: After the page is loaded and all connections are closed, when I scroll to the top of the log window, I am in the middle of the page load, not at the beginning of it. Just to make sure that I am doing this correctly, I clear the browser cache, close and then restart Prox to reset the numbering of requests and responses, open the log window, click the link to the page I want to log, pause logging after the page has finished loading, and then scroll to the top of the log window. Instead of seeing 'New Message Log Window' at the top, followed closely by '+++GET 1+++', I am seeing a few header lines followed by '+++RESP 13+++'.

Is there a way to increase the log window buffer size or, better yet, capture the log to a file as it is being generated? If neither of those are possible, is there another possible workaround? Thanks for your help, and apologies if this question has already been answered somewhere.

Jeff

[JJoe]

(Oct. 17, 2010 03:20 AM)Aljan Darkmoon Wrote:  Is there a way to increase the log window buffer size or, better yet, capture the log to a file as it is being generated?

I don't have either.

(Oct. 17, 2010 03:20 AM)Aljan Darkmoon Wrote:  If neither of those are possible, is there another possible workaround?

I saw your post at the list. I don't have a solution other than the obvious, get the first half and then reload to get the second.
The log window's right click menu allows logging to be paused.
Use the browser to stop loading.
You may be able to use the Proxomitron to stop loading at a specific spot.

All the addresses may be found in http://local.ptron/.pinfo/urls/ .
MaxCapURL in the cfg file can increase the number of addresses shown by http://local.ptron/.pinfo/urls/ .
Loading just one file and then viewing the log may help.


HTH

[Aljan Darkmoon]

Questions first (comments after):

(Oct. 17, 2010 08:50 PM)JJoe Wrote:  You may be able to use the Proxomitron to stop loading at a specific spot.

Abort button? Special filter? Not sure what you have in mind...

JJoe Wrote:Loading just one file and then viewing the log may help.

By 'file', do you mean 'link', as in "go to http://local.ptron/.pinfo/urls/ , go down the list of addresses that the test page (the one I'm trying to log) loaded, load one address at a time and view the log for each"?


Comments:

JJoe Wrote:I saw your post at the list.

I figured, since I saw your recent posts there. Thought I'd try here, too, and add more detail.

JJoe Wrote:I don't have a solution other than the obvious, get the first half and then reload to get the second.

Thought of that, but also thought about how the GET, RESP, and filter match ID numbers will not match up between the two halves, even if I restart Prox to reset the numbering (though that may help some). However, it may still be possible to trace what's happening if the overlap between the two halves is large enough. I'll give that a try.

JJoe Wrote:The log window's right click menu allows logging to be paused.
Use the browser to stop loading.

Thought of pausing the log (with CTRL+P), but not of stopping the browser. That may be the best, because it may not be possible to change focus from the browser to the log window quickly enough to pause it in time.

JJoe Wrote:All the addresses may be found in http://local.ptron/.pinfo/urls/ .
MaxCapURL in the cfg file can increase the number of addresses shown.

That will help, but I also need to see URL-killer filter matches to find out if anything essential is being blocked. /k-killed connections apparently do not show up in the list of addresses (I just tested that out on a couple of pages that link to domains I have blocked).

JJoe Wrote:HTH

It's a start, and it's more than I had before. Thank you.

[JJoe]

(Oct. 17, 2010 11:16 PM)Aljan Darkmoon Wrote:  

(Oct. 17, 2010 08:50 PM)JJoe Wrote:  You may be able to use the Proxomitron to stop loading at a specific spot.

Abort button? Special filter? Not sure what you have in mind...

An expression like

Code:

$SET(StopIt=$DTM(c))$TST(StopIt=[#nnn:*])$RDIR(http://local.ptron/killed.gif)

used in a filter or list. "nnn" is the connection number that you wish to stop at. The browser will probably make request beyond "nnn" but the Proxomitron will redirect all requests after "nnn".

(Oct. 17, 2010 11:16 PM)Aljan Darkmoon Wrote:  

JJoe Wrote:Loading just one file and then viewing the log may help.

By 'file', do you mean 'link', as in "go to http://local.ptron/.pinfo/urls/ , go down the list of addresses that the test page (the one I'm trying to log) loaded, load one address at a time and view the log for each"?

Sort of, my hope is that you don't need to examine every request. Also, you may have to fake some data to make the server reply but I didn't say I had the solution.

(Oct. 17, 2010 11:16 PM)Aljan Darkmoon Wrote:  

JJoe Wrote:All the addresses may be found in http://local.ptron/.pinfo/urls/ .
MaxCapURL in the cfg file can increase the number of addresses shown.

That will help, but I also need to see URL-killer filter matches to find out if anything essential is being blocked. /k-killed connections apparently do not show up in the list of addresses (I just tested that out on a couple of pages that link to domains I have blocked).

\k redirects to a "local.ptron" address and "local.ptron" addresses do not show in http://local.ptron/.pinfo/urls/.
However, assuming that Proxo is listening on port 8080, "local.ptron" resolves to 127.0.0.1:8080. Redirects to 127.0.0.1:8080 do show in http://local.ptron/.pinfo/urls/.
So, if you're willing to modify your set.

HTH

http://127.0.0.1:8080/proxylogo.jpg