Post Reply 
Nail down this bug?
Oct. 16, 2010, 08:31 PM (This post was last modified: Oct. 16, 2010 10:22 PM by JJoe.)
Post: #16
RE: Nail down this bug?
(Oct. 16, 2010 05:54 PM)sidki3003 Wrote:  It's apparently only about the .ptron file, the lists don't seem to be affected.

Quote:Any list call after a "negate string positional variable mess" seems to initialize all positional variables.

I was wrong. Something else in the web page filters was 'fixing' things.

TEST 1

Add: (^z)\0nevermatch(^)|$TST(\0=*) to Bypass List.txt.
Load: Google
Result: Web Page incorrectly bypassed.
Remove: Bypass entry



TEST 2

Add following filters

Code:
[Patterns]
Name = "A -- correct: $TST(\0=*) does not match"
Active = TRUE
Limit = 256
Match = "$TST(\0=*)"
Replace = "\k"

Name = "E --  use list to create problem"
Active = TRUE
Limit = 256
Match = "$LST(Bypass)"

Name = "Es --  use list to create problem"
Active = TRUE
Limit = 256
Match = "$LST(Bypass-List)"

Add: (^z)\0nevermatch(^) to Bypass List.txt.
Load: Google
Result: Web Page incorrectly 'killed'.



TEST 3

Enable: Filter A
Disable: Filter E and Filter Es
Add: (^z)\0nevermatch(^) to Bypass List.txt.
Load: Google
Result: Web Page loads as expected.

This test 3 shows that the \0 problem (created in the Bypass list) was 'fixed' by the Proxomitron before the Web Page Filter A tested the variable.

Remove: Bypass entry and filters
Note: Filter E is for Scott's set. Filter Es is for sidki's set.

Edit:

(Oct. 16, 2010 09:21 AM)whenever Wrote:  
(Oct. 16, 2010 04:48 AM)JJoe Wrote:  However, after filter "D" fails, filter "A" finds "$TST(\0=*)" to be true and matches.

Well, finally a serious bug found: a positional var lives between different filters. Thumbs Up

Sidki mentioned another quirk about positional var before, which seems related: http://prxbx.com/forums/showthread.php?t...2#pid11062

Sidki also found this one. I'm just trying to document it and keep things moving.
Add Thank You Quote this message in a reply
Oct. 17, 2010, 11:23 AM (This post was last modified: Oct. 17, 2010 11:25 AM by sidki3003.)
Post: #17
RE: Nail down this bug?
(Oct. 16, 2010 08:31 PM)JJoe Wrote:  
(Oct. 16, 2010 05:54 PM)sidki3003 Wrote:  It's apparently only about the .ptron file, the lists don't seem to be affected.

What i was trying to say here is that i didn't spot any discussed potentially troublesome expressions in any of the lists.


Quote:This test 3 shows that the \0 problem (created in the Bypass list) was 'fixed' by the Proxomitron before the Web Page Filter A tested the variable.

This shows again that URL inspection and content inspection are distinct - also timely separated - stages.
"Bypass List.txt" normally gets scanned during URL inspection.

Another consequence of this separation is that you can't evaluate positional variables in the matching/replace expressions, which you have set in the "URL match" field of the same filter (header or web).
Add Thank You Quote this message in a reply
Oct. 18, 2010, 07:50 PM
Post: #18
RE: Nail down this bug?
(Oct. 16, 2010 05:54 PM)sidki3003 Wrote:  OK. For now i've used an online tool ( http://gskinner.com/RegExr/ ) - which does do multi-line - with whenever's RegEx

were you able to find a "non-online" tool for this mult-line RegEx thingie?
Add Thank You Quote this message in a reply
Oct. 19, 2010, 12:21 AM
Post: #19
RE: Nail down this bug?
I didn't look for one, actually, now that the MergeMe files are ready.
But yes, in the long run it would be nice to have usable Windows program at hand...
Add Thank You Quote this message in a reply
Oct. 19, 2010, 02:12 AM
Post: #20
RE: Nail down this bug?
The lite version of NoteTab does do multiline regex search.
Add Thank You Quote this message in a reply
Oct. 19, 2010, 03:41 PM (This post was last modified: Oct. 19, 2010 04:31 PM by sidki3003.)
Post: #21
RE: Nail down this bug?
JJoe, do you like to add a short chapter (11) about proper handling of discussed issue to Techniques.txt?
Add Thank You Quote this message in a reply
Oct. 19, 2010, 10:37 PM
Post: #22
RE: Nail down this bug?
(Oct. 19, 2010 03:41 PM)sidki3003 Wrote:  JJoe, do you like to add a short chapter (11) about proper handling of discussed issue to Techniques.txt?

A good idea. This situation is similar to chapter 1.
I'll try to put something together before bedtime.
You may use some, all, or none of it.
Add Thank You Quote this message in a reply
Oct. 19, 2010, 11:01 PM (This post was last modified: Oct. 19, 2010 11:10 PM by sidki3003.)
Post: #23
RE: Nail down this bug?
Okay, great!
Then we better make it chapter 2 and move the other chapters one down?

edit: OTOH, there is a transition between chapter 1 and 2 ($IHDR/$OHDR)...
Add Thank You Quote this message in a reply
Oct. 19, 2010, 11:44 PM
Post: #24
RE: Nail down this bug?
(Oct. 19, 2010 11:01 PM)sidki3003 Wrote:  Okay, great!
Then we better make it chapter 2 and move the other chapters one down?

edit: OTOH, there is a transition between chapter 1 and 2 ($IHDR/$OHDR)...

I think, begin with workarounds (show cause for 'standard') and then use workarounds in all that follows.

So I don't have a problem with either separating the issues discussed in the current chapters 1 and 2 or beginning with the (^....)\x issue.
Add Thank You Quote this message in a reply
Oct. 20, 2010, 03:13 AM
Post: #25
RE: Nail down this bug?
What about

Code:
-------------------------------------------------------------------------------
?   A positional variable that too closely follows a negated expression may be
    incorrectly found to have value in the expressions and filters that follow.
    To cover this, always place positional variables that are next to and follow
    negated expressions in parentheses.

Examples:
(^[a-z])(\0)
(^\\$TST(\3))(\9)
$TST(keyword=(^*.i_adtag:[#*:0].)(\7))
(^([^/]++.|)$TST(uDom)(^.))(\9)

?
Add Thank You Quote this message in a reply
Oct. 20, 2010, 04:14 PM
Post: #26
RE: Nail down this bug?
We can be more exact than "too closely follows". -> "directly follows"
Also mention that we are in capturing stage (vs. replace)?

(This bug is - as whenever already mentioned - probably related to another issue where \X doesn't find anything to capture. So i assume that the here discussed problem only appears when \X is in a similar interim state: Nothing to match after proceeding to the end of expression, but no chance to clean up. However, that would go most likely beyond the scope of this document.)

The examples should show the simple case, and then only variations that show further aspects. I think your example, showing the issue within a global var test, is important.

In the other chapters the term "local variable" is used. I'll replace that with "positional variable".


Code:
-------------------------------------------------------------------------------
2   A capturing positional variable that directly follows a negated expression
    may be incorrectly found to have value in the expressions and filters that
    follow.  To cover this, place positional variables that are next to and
    follow negated expressions in parentheses.

Examples:
(^[a-z])(\0)
(^foo(^bar))(\9)
$TST(foo=(^*bar)(\7))

?
Add Thank You Quote this message in a reply
Oct. 20, 2010, 07:03 PM (This post was last modified: Oct. 20, 2010 07:05 PM by JJoe.)
Post: #27
RE: Nail down this bug?
(Oct. 20, 2010 04:14 PM)sidki3003 Wrote:  ?

Better. I hadn't considered the "capturing stage (vs. replace)" angle.
I did consider "directly follows", also ok. Help uses
Quote:directly after a set of parenthesis with no spaces in-between as in "(foo*bar)\1"
Our second sentence, using "next to", and the examples should handle the "with no spaces in-between" part.

$TST() and the rest seemed unnecessary, so I removed them.

"positional variable" or "local variable", I don't know which is best.

Edit: Will be gone for a few hours.
Add Thank You Quote this message in a reply
Oct. 20, 2010, 11:04 PM
Post: #28
RE: Nail down this bug?
(Oct. 20, 2010 04:14 PM)sidki3003 Wrote:  (This bug is - as whenever already mentioned - probably related to another issue where \X doesn't find anything to capture. So i assume that the here discussed problem only appears when \X is in a similar interim state: Nothing to match after proceeding to the end of expression, but no chance to clean up. However, that would go most likely beyond the scope of this document.)

Looks like I'll have something to say about that one, I think.
Add Thank You Quote this message in a reply
Oct. 21, 2010, 03:30 PM
Post: #29
RE: Nail down this bug?
Okay, i'll attach the current Techniques.txt.
The online version is missing two sections by now (2 and 11). I'll update the online docs when i upload the new config.


Attached File(s)
.txt  Techniques.txt (Size: 6.56 KB / Downloads: 265)
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: