Post Reply 
User-Agents.ptxt update 2013-12-26
Dec. 27, 2013, 04:09 AM (This post was last modified: Dec. 29, 2013 05:48 PM by JJoe.)
Post: #1
User-Agents.ptxt update 2013-12-26
2013-12-29 removed some errant spaces.

"Fake" strings may need to be updated.

Download the file with the Proxomitron in bypass!
Do not build the list with copy and paste from this forum's code block! Some necessary spaces may be lost.

Code:
###############################################################################
#  NoHash
#  NOADDURL $LST(User-Agents) -- Naoko 4.5
#  Manage listed User-Agents
#
#  List for the header filter "User-Agent: Manage listed Strings".
#
#  sidki 2002-09-03
#  Last updated 2010-10-03
#  JJoe 2012-07-06 Added NoHash; Add IE9
#       ; Allow trident, chrome, safari, and presto values due to picky servers
#  JJoe 2013-12-26 Add IE10, IE11
#  JJoe 2013-12-29 removed some errant spaces.
#
#
###############################################################################


## |||||||||||||||||||||||||||||| Kill connection |||||||||||||||||||||||||||||

## Winamp Lyrics Plugin
## ----------------------------------------------------------------------------
UtilMind HTTPGet    $SET(1=\0$RDIR(http://local.ptron/killed.gif))


## ||||||||||||||||||||||||||||| Bypass webfilters ||||||||||||||||||||||||||||

## Flash (POST, application/x-fcs)
## ----------------------------------------------------------------------------
Shockwave Flash            $SET(1=\0$FILTER(0))

## XP Search Assistant (text/xml)
## ----------------------------------------------------------------------------
SCAgent                $SET(1=\0$FILTER(0))

## WinBatch (text/plain)
## ----------------------------------------------------------------------------
WinBatch Internet Extender Ver:    $SET(1=\0$FILTER(0))


## ||||||||||||||||||||||||||| "Correct" user-agent |||||||||||||||||||||||||||

## If MSIE:
## ----------------------------------------------------------------------------
#Mozilla/4.0(^$TST(keyword=*.f_ua_(^[a-z]++ie)*)) \(compatible; MSIE(^ 5.01)\2;
#  $SET(1=Mozilla/4.0 (compatible; MSIE\2; Windows NT 6.1))

#IE11
Mozilla/5.0 (^$TST(keyword=*.f_ua_(^[a-z]++ie)*))
  \(*Trident/([7-9].0)\4*rv: (1[1-9].0)\3
  $SET(1=Mozilla/5.0 \(Windows NT 6.1; Trident/\4; rv \3\) like Gecko)

#IE10 and below
Mozilla/([4-9].0)\3 (^$TST(keyword=*.f_ua_(^[a-z]++ie)*))
  \(compatible; MSIE(^ 5.01)\2;(*(trident/[4-9].0)\4[;\)])+
  $SET(1=Mozilla/\3 \(compatible; MSIE\2; Windows NT 6.1; \4\))


## If Firefox, SeaMonkey, Chrome, Safari:
## ----------------------------------------------------------------------------
#Mozilla/5.0(^$TST(keyword=*.f_ua_(^[a-z]++moz)*))(*(; rv:*Gecko/[#*:*]
#  ( Firefox/[#*:*].[#*:*]$SET(3=.9.9)|)|\)
#  (*\s(Gecko/[#*:*]|AppleWebKit/(*.)+{1}$SET(3=99))|))\2
#  [0-9.]+( $NEST(\(,*gecko*,\))|)\4|$SET(2=\)))
#  $SET(1=Mozilla/5.0 \(Windows; U; Windows NT 6.1; en-US\2\3\4)

Mozilla/5.0(^$TST(keyword=*.f_ua_(^[a-z]++moz)*))
  (*\()+{1}Windows nt
  (*(; rv:*Gecko/[#*:*]
  ( Firefox/[#*:*].[#*:*]$SET(3=.9.9)|)|\)
  (*\s(Gecko/[#*:*]|AppleWebKit/(*.)+{1}$SET(3=99))|))\2
  [0-9.]+( $NEST(\(,*gecko*,\))|)\4|$SET(2=\)))
  (( (Chrome/|Safari/)[0-9][0-9.]+)+)\5
  $SET(1=Mozilla/5.0 \(Windows NT 6.1\2\3\4\5)

Mozilla/5.0(^$TST(keyword=*.f_ua_(^[a-z]++moz)*))
  (*\()+{1}Windows;
  (*(; rv:*Gecko/[#*:*]
  ( Firefox/[#*:*].[#*:*]$SET(3=.9.9)|)|\)
  (*\s(Gecko/[#*:*]|AppleWebKit/(*.)+{1}$SET(3=99))|))\2
  [0-9.]+( $NEST(\(,*gecko*,\))|)\4|$SET(2=\)))
  (( (Chrome/|Safari/)[0-9][0-9.]+)+)\5
  $SET(1=Mozilla/5.0 \(Windows; U; Windows NT 6.1\2\3\4\5)

## If Opera:
## ----------------------------------------------------------------------------
#Opera/(^$TST(keyword=*.f_ua_(^[a-z]++op)*))(\w)\2
#  (*(\sVersion/[#*:*].[0-9]$SET(4=9))\3|)
#  $SET(1=Opera/\2 (Windows NT 6.1; U)\3\4)

Opera/(^$TST(keyword=*.f_ua_(^[a-z]++op)*))(\w)\2
  ((*(\s(Presto/[0-9.][0-9.]+|Version/[#*:*].[0-9]$SET(#=9)))\#)+|)
  $SET(1=Opera/\2 \(Windows NT 6.1; U\)\@)

## |||||||||||||||||||||||||||||| Fake user-agent |||||||||||||||||||||||||||||

## Keywords:
##   f_ua_ie            -> fake MSIE
##   f_ua_iemoz         -> fake MSIE, if not IE or Mozilla
##   f_ua_moz           -> fake Mozilla
##   f_ua_mozie         -> fake Mozilla, if not IE or Mozilla
##   f_ua_op            -> fake Opera
##   f_ua_opie          -> fake Opera, if not IE or Opera
##   f_ua_sp            -> fake Space Bison
##   f_ua_gbot          -> fake Googlebot
##   f_ua_§Anything     -> fake unescaped "Anything"
## ----------------------------------------------------------------------------
$TST(keyword=*.f_ua_(
  ie.(^$TST(hOrigUA=*MSIE(^*Opera)*))
  $SET(1=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 6.1))
|iemoz.(^$TST(hOrigUA=*(Gecko/|MSIE(^*Opera))*))
  $SET(1=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 6.1))
|moz.(^$TST(hOrigUA=*Gecko/*))
  $SET(1=Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0)
|mozie.(^$TST(hOrigUA=*(Gecko/|MSIE(^*Opera))*))
  $SET(1=Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0)
|op.(^$TST(hOrigUA=*Opera*))
  $SET(1=Opera/9.20 (Windows NT 6.1; U; en))
|opie.(^$TST(hOrigUA=*(Opera|MSIE)*))
  $SET(1=Opera/9.20 (Windows NT 6.1; U; en))
|sp.
  $SET(1=Space Bison/0.02 [fu] (Win67; X; SK))
|gbot.
  $SET(1=Googlebot/3.0 Beta (+http://www.googlebot.com/bot.html))
|§\2.$SET(1=$UESC(\2))
)*)


Attached File(s)
.ptxt  User-Agents.ptxt (Size: 4.96 KB / Downloads: 506)
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to JJoe for this post:
Belarathon
Dec. 27, 2013, 04:44 AM
Post: #2
RE: User-Agents.ptxt update 2013-12-26
That fixed it. Thanks again.
Add Thank You Quote this message in a reply
Dec. 27, 2013, 12:57 PM
Post: #3
RE: User-Agents.ptxt update 2013-12-26
(Dec. 27, 2013 04:44 AM)Belarathon Wrote:  That fixed it. Thanks again.

Oops....this breaks Google search. It'll find results, but when you click on links, the destination site is never reached.

Also, I just noticed this morning, but Youtube now always displays a "You need Adobe Flash Player..." banner before the video starts.
Add Thank You Quote this message in a reply
Dec. 27, 2013, 03:43 PM
Post: #4
RE: User-Agents.ptxt update 2013-12-26
(Dec. 27, 2013 12:57 PM)Belarathon Wrote:  Oops....this breaks Google search. It'll find results, but when you click on links, the destination site is never reached.

Also, I just noticed this morning, but Youtube now always displays a "You need Adobe Flash Player..." banner before the video starts.

Granted that I have IE10 Win7 and IE11's javascript properties have changed and "proxjs-full.js" probably needs to be updated, I can't replicate.
Google and Youtube work as before while sending "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko".

What UA is your Proxomitron sending?

Does the Proxomitron's log window show the headers for the destination site after clicking at google? Maybe IE11 has some sort of built-in Phishing and Malware protection and that server doesn't like something.
Add Thank You Quote this message in a reply
Dec. 27, 2013, 08:50 PM (This post was last modified: Dec. 28, 2013 02:32 AM by Belarathon.)
Post: #5
RE: User-Agents.ptxt update 2013-12-26
(Dec. 27, 2013 03:43 PM)JJoe Wrote:  What UA is your Proxomitron sending?
New list is sending:

User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko

(Dec. 27, 2013 03:43 PM)JJoe Wrote:  Does the Proxomitron's log window show the headers for the destination site after clicking at google?

If I go to Google.com and search for "test", this is what I get after clicking on the first search return (apologies if I pasted far too much)

Code:
*** Log Reset ***
GET 240 : Keywords (List): .a_rdlink.f_cookie.
JumpTo: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=Dee9UqEti4-QB4jIgaAG&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0
240 JUMP to google complete off
BlockList 240: in User-Agents, line 48
GET 240 : Time: 14:46:44::458

+++GET 240+++
GET /url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=Dee9UqEti4-QB4jIgaAG&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0 HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko
Accept-Encoding: gzip,deflate
Host: www.google.com
Connection: keep-alive
Cookie: PREF=ID=3003added0032123:FF=4:NR=100:CR=2:SG=0
+++CLOSE 240+++
GET 241 : Keywords (List): .a_rdlink.f_cookie.
BlockList 241: in User-Agents, line 48
GET 241 : Time: 14:46:44::480

+++GET 241+++
GET /url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=Dee9UqEti4-QB4jIgaAG&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0 HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko
Accept-Encoding: gzip,deflate
Host: www.google.com
Connection: keep-alive
Cookie: PREF=ID=3003added0032123:FF=4:NR=100:CR=2:SG=0
RESP 241 : Cache-Control replaced: no-cache, must-revalidate
RESP 241 : Expires killed: Fri, 01 Jan 1990 00:00:00 GMT
RESP 241 : Pragma no-cache stripped
RESP 241 : Time: 14:46:44::599

+++RESP 241+++
HTTP/1.1 200 OK
Date: Fri, 27 Dec 2013 20:46:44 GMT
Cache-Control: max-age=1
X-Frame-Options: ALLOWALL
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: gws
Content-Length: 513
X-XSS-Protection: 1; mode=block
Alternate-Protocol: 80:quic
|.*.URL-ID: (241) http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=Dee9UqEti4-QB4jIgaAG&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0
Match 241: Top All Mark: Start     04.07.11 (multi) [sd] (d.r)
Match 241: Top All Mark: End     06.12.25 [sd] (d.r)
Match 241: Top JS Mark: Start     10.10.09 (multi) [sd] (d.r)
Match 241: Top JS: Mark End     10.10.09 [sd] (d.r)
Match 241: Top HTML Mark: Start     09.06.12 (multi) [sd] (d.r)
Match 241: Top Sniff: HTML Content: HTML     10.10.16 (multi) [sd] (d.1)
Match 241: Top Mark: HTML - Fix Tags     09.06.28 (multi) [sd] (d.r)
Match 241: Top HTML Mark: End     07.10.24 [sd] (d.r)
Match 241: Header Top Mark: Start - Fix <head>     10.10.16 (multi) [sd] (d.r)
Match 241: Header Top Add: Initial JS Code     09.11.01 (ccw! !mos) [...] (d.r)
Match 241: Header Top Mark: End     07.09.06 (multi) [sd] (d.r)
Match 241: Header Bot Mark: Start - Fix </head>     09.06.29 (multi) [sd] (d.r)
Match 241: Google Search: Remove Ad Blocks     11.11.21 [sd ku] (d.s)
Match 241: Header Bot Add: Default Script/Style Type if Missing     07.08.31 [sd] (d.1)
Match 241: Header Bot Add: Navigation Links     09.07.04 [sd] (d.1)
Match 241: Header Bot Mark: End     07.11.02 [sd] (d.r)
Match 241: <body> Mark: Start     09.06.20 (multi) [sd] (d.r)
Match 241: Body Add: JS Code     09.06.13 (ccw! !nn !mos) [...] (d.r l.3)
Match 241: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
Match 241: Remove: Specific Post-HTML Blocks     10.10.16 (multi) [sd] (d.2 l.3)
Match 241: HTML Bottom Mark: Start - Close open Tags     09.05.17 (multi) [sd] (d.r)
Match 241: HTML Bottom Mark: Start - Close open Tags     09.05.17 (multi) [sd] (d.r)
Match 241: Bottom Add: Final JS Code     09.06.13 (ccw! !mos) [...] (d.r)
Match 241: Bottom Mark: End     09.05.08 [sd] (d.r)
Match 241: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
Match 241: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
+++CLOSE 241+++


It wasn't my intention to open up a can of worms. The only real issue I was having was with YT. If I leave the old UA list in place, I can avoid my youtube issues by disabling the "Block/Modify: Sel. JS Properties 07.04.02 [sd] (d.2)" rule. It doesn't seem to adversely affect Proxomitron's blocking abilities. At least, nothing as far as the eye can see. The new list must be suspect, because this trick doesn't work with it. (at least as far as my feeble reasoning abilities can discern.
Add Thank You Quote this message in a reply
Dec. 28, 2013, 02:55 AM
Post: #6
RE: User-Agents.ptxt update 2013-12-26
(Dec. 27, 2013 08:50 PM)Belarathon Wrote:  It wasn't my intention to open up a can of worms.

I know but lets fix what we can and at least try to understand what we can't fix.

(Dec. 27, 2013 08:50 PM)Belarathon Wrote:  New list is sending:

User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko

As expected. I see little need to advertise more than necessary.

(Dec. 27, 2013 08:50 PM)Belarathon Wrote:  If I go to Google.com and search for "test", this is what I get after clicking on the first search return

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=Dee9UqEti4-QB4jIgaAG&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0
?!

You shouldn't be seeing this pseudo url. Google is tracking you. Have you disabled the "Google: Multi URL Untangler" filter?

Google's code for this link is

Code:
<a href="http://www.speedtest.net/" onmousedown="return rwt(this,'','','','1','AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw','','0CCsQFjAA','','',event)">

The onmousedown part allows google's script to hijack you to the psuedo url. The psuedo url's code is suppose to redirect you to speedtest but the sidki set is breaking the redirect. We tend to not appreciate such hijinks.

"Google: Multi URL Untangler" filter is supposed to remove the onmousedown part.

(Dec. 27, 2013 08:50 PM)Belarathon Wrote:  "Block/Modify: Sel. JS Properties 07.04.02 [sd] (d.2)"

isn't a required filter but it isn't necessary to disable everywhere. Use the keyword "a_jsprop" in Exceptions-U.
Add Thank You Quote this message in a reply
Dec. 28, 2013, 04:02 AM (This post was last modified: Dec. 28, 2013 04:30 AM by Belarathon.)
Post: #7
RE: User-Agents.ptxt update 2013-12-26
(Dec. 28, 2013 02:55 AM)JJoe Wrote:  You shouldn't be seeing this pseudo url. Google is tracking you. Have you disabled the "Google: Multi URL Untangler" filter?
"Google: Multi URL Untangler" filter is supposed to remove the onmousedown part.

Yes, that's enabled. Other than block/modify: Sel. JS Properties,
I have Block/Modify: Sel. JS Methods 10.09.07 [sd] (d.r) disabled, because it interferes with new tabs opening when clicking certain links (rather than right clicking on a link and selecting "open in new tab." Now that I look at it, I see that the rule is appended with an "r," which I now realize must be "required." Would that be why my butt is hanging out on Google?

Edit: Yep, that must have been why. I just repeated the search, and the Proxo log was filled with the Google URL Untangler filter.

Well, that was premature. I just now put the new UA list in, and google is broken again, and now no mention of the URL Untangler. I'll post log with, and without new list below.
Add Thank You Quote this message in a reply
Dec. 28, 2013, 04:31 AM (This post was last modified: Dec. 28, 2013 04:41 AM by Belarathon.)
Post: #8
RE: User-Agents.ptxt update 2013-12-26
With new UA list:

Code:
*** Log Reset ***
GET 601 : Keywords (List): .a_rdlink.f_cookie.
JumpTo: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=IlO-UvXMLsWtkAfMkYBQ&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&bvm=bv.58187178,d.eW0&complete=0
601 JUMP to google complete off
BlockList 601: in User-Agents, line 48
GET 601 : Time: 22:27:17::128

+++GET 601+++
GET /url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=IlO-UvXMLsWtkAfMkYBQ&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&bvm=bv.58187178,d.eW0&complete=0 HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko
Accept-Encoding: gzip,deflate
Host: www.google.com
Connection: keep-alive
Cookie: PREF=ID=3003added0032123:FF=4:NR=100:CR=2:SG=0
+++CLOSE 601+++
GET 602 : Keywords (List): .a_rdlink.f_cookie.
BlockList 602: in User-Agents, line 48
GET 602 : Time: 22:27:17::150

+++GET 602+++
GET /url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=IlO-UvXMLsWtkAfMkYBQ&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&bvm=bv.58187178,d.eW0&complete=0 HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko
Accept-Encoding: gzip,deflate
Host: www.google.com
Connection: keep-alive
Cookie: PREF=ID=3003added0032123:FF=4:NR=100:CR=2:SG=0
RESP 602 : Cache-Control replaced: no-cache, must-revalidate
RESP 602 : Expires killed: Fri, 01 Jan 1990 00:00:00 GMT
RESP 602 : Pragma no-cache stripped
RESP 602 : Time: 22:27:17::350

+++RESP 602+++
HTTP/1.1 200 OK
Date: Sat, 28 Dec 2013 04:27:16 GMT
Cache-Control: max-age=1
X-Frame-Options: ALLOWALL
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: gws
Content-Length: 513
X-XSS-Protection: 1; mode=block
Alternate-Protocol: 80:quic
|.*.URL-ID: (602) http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCkQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=IlO-UvXMLsWtkAfMkYBQ&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&bvm=bv.58187178,d.eW0&complete=0
Match 602: Top All Mark: Start     04.07.11 (multi) [sd] (d.r)
Match 602: Top All Mark: End     06.12.25 [sd] (d.r)
Match 602: Top JS Mark: Start     10.10.09 (multi) [sd] (d.r)
Match 602: Top JS: Mark End     10.10.09 [sd] (d.r)
Match 602: Top HTML Mark: Start     09.06.12 (multi) [sd] (d.r)
Match 602: Top Sniff: HTML Content: HTML     10.10.16 (multi) [sd] (d.1)
Match 602: Top Mark: HTML - Fix Tags     09.06.28 (multi) [sd] (d.r)
Match 602: Top HTML Mark: End     07.10.24 [sd] (d.r)
Match 602: Header Top Mark: Start - Fix <head>     10.10.16 (multi) [sd] (d.r)
Match 602: Header Top Add: Initial JS Code     09.11.01 (ccw! !mos) [...] (d.r)
Match 602: Header Top Mark: End     07.09.06 (multi) [sd] (d.r)
Match 602: Header Bot Mark: Start - Fix </head>     09.06.29 (multi) [sd] (d.r)
Match 602: Google Search: Remove Ad Blocks     11.11.21 [sd ku] (d.s)
Match 602: Header Bot Add: Default Script/Style Type if Missing     07.08.31 [sd] (d.1)
Match 602: Header Bot Add: Navigation Links     09.07.04 [sd] (d.1)
Match 602: Header Bot Mark: End     07.11.02 [sd] (d.r)
Match 602: <body> Mark: Start     09.06.20 (multi) [sd] (d.r)
Match 602: Body Add: JS Code     09.06.13 (ccw! !nn !mos) [...] (d.r l.3)
Match 602: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
Match 602: Remove: Specific Post-HTML Blocks     10.10.16 (multi) [sd] (d.2 l.3)
Match 602: HTML Bottom Mark: Start - Close open Tags     09.05.17 (multi) [sd] (d.r)
Match 602: HTML Bottom Mark: Start - Close open Tags     09.05.17 (multi) [sd] (d.r)
Match 602: Bottom Add: Final JS Code     09.06.13 (ccw! !mos) [...] (d.r)
Match 602: Bottom Mark: End     09.05.08 [sd] (d.r)
Match 602: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
Match 602: <html><body>: Mark First - Remove Dupes     09.06.28 (multi) [sd] (d.r)
+++CLOSE 602+++
GET 603 : Keywords (List): .a_rdlink.f_cookie.
BlockList 603: in User-Agents, line 48
GET 603 : Time: 22:27:17::542

+++GET 603+++
GET /favicon.ico HTTP/1.1
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv 11.0) like Gecko
Host: www.google.com
Connection: keep-alive
Cookie: PREF=ID=3003added0032123:FF=4:NR=100:CR=2:SG=0
RESP 603 : Age killed: 45005
RESP 603 : Expires killed: Sat, 04 Jan 2014 15:57:12 GMT
RESP 603 : Vary killed: Accept-Encoding
RESP 603 : Time: 22:27:17::772

+++RESP 603+++
HTTP/1.1 200 OK
Content-Type: image/x-icon; PrxMsg: Filter Image
Last-Modified: Tue, 14 Aug 2012 15:19:23 GMT
Date: Fri, 27 Dec 2013 15:57:12 GMT
X-Content-Type-Options: nosniff
Server: sffe
X-XSS-Protection: 1; mode=block
Cache-Control: public, max-age=691200
Alternate-Protocol: 80:quic
Transfer-Encoding: chunked
|.*.URL-ID: (603) http://www.google.com/favicon.ico
Match 603: Top All Mark: Start     04.07.11 (multi) [sd] (d.r)
Match 603: Top Sniff: Sel. File Types     10.10.18 [sd] (d.1)
Match 603: Protect: Sel. File Types     07.07.15 [sd] (d.r)
+++CLOSE 603+++



With old UA List:

I've attached the old log; it was too big to post. It's working as it should, the URL Untangler shows up multiple times.


To further update, the Google URL Untangler works with both block/modify select js properties and select js methods disabled with the old UA list; not so with the new one. Something about the new UA list is apparently breaking the untangler.


Attached File(s)
.log  old.log (Size: 89.3 KB / Downloads: 296)
Add Thank You Quote this message in a reply
Dec. 28, 2013, 05:43 AM
Post: #9
RE: User-Agents.ptxt update 2013-12-26
Use \(d.r to find all the required filters and make sure the latest version of each is enabled. We can disable with Exceptions-U, if needed or desired.

(Dec. 28, 2013 04:31 AM)Belarathon Wrote:  To further update, the Google URL Untangler works with both block/modify select js properties and select js methods disabled with the old UA list; not so with the new one. Something about the new UA list is apparently breaking the untangler.

Strange. The filter only references page html. So it seems more likely, google is sending different code to you for different UAs?

Problem with Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) is that it is too unique and without rendering engine.

To test another UA, this Exception-U entry

Code:
$SET(keyword=$GET(keyword)f_ua_§Mozilla/5%2e0 (compatible; MSIE 10%2e0; Windows NT 6%2e1; Trident/6%2e0).)(^)
~$SET(keyword=$GET(keyword)f_ua_§Mozilla/5%2e0 (compatible; MSIE 10%2e0; Windows NT 6%2e1; Trident/6%2e0).)(^)

should give you a IE10 UA


Code:
[Patterns]
Name = "Google: Multi URL Untangler     11.10.10 (multi) [sd jjoe] (d.s)"
Active = TRUE
Multi = TRUE
URL = "$TST(hCT=*html)(*.)+{1}google."
Bounds = "<a\s*>"
Limit = 512
Match = "<a(\s*)\7("
        "(href=)\4$AV(/url\?*=(https+(://|%3a%2f%2f))\5("
        "\6\&(amp;)+[a-z][a-z]+=*|\6"
        "))$SET(2=\4"$UESC(\5\6)")"
        "| onmousedown=$AV(return (clk|rwt|hlprwt) \(*)$SET(1=Js)"
        ")(\3)"
        "&$TST(\7=(\#\sclass=$AV(\8)$SET(s=$GET(s)&#x20;\8))+\#)"
Replace = "<a class="Pr0XPref\1$GET(s)$SET(s=)"\@\2\3"
Add Thank You Quote this message in a reply
Dec. 28, 2013, 11:13 AM (This post was last modified: Dec. 28, 2013 04:15 PM by JJoe.)
Post: #10
RE: User-Agents.ptxt update 2013-12-26
I am NOT an html programmer. Can you please tell me how I use this information above to modify the user agent proxom sends to the server?

Edit by JJoe: trimmed topic by removed quote of first post.
Add Thank You Quote this message in a reply
Dec. 28, 2013, 12:44 PM
Post: #11
RE: User-Agents.ptxt update 2013-12-26
the list above replaces the file that you extracted when you first "installed" Proxomitron...

if you are new to Proxomitron, you should be starting with something "easier" than user-agent strings and use your browser user-agent settings instead of Proxomitron...

you don't have to be an HTML programmer to use Proxomitron, but YES, you DO have to be familiar with HTML code if you want to "tweak" Proxomitron's config files...

so if you aren't "familiar" with HTML code, then Proxomitron probably isn't your cup of tea...
Add Thank You Quote this message in a reply
Dec. 28, 2013, 01:39 PM
Post: #12
RE: User-Agents.ptxt update 2013-12-26
(Dec. 28, 2013 05:43 AM)JJoe Wrote:  Use \(d.r to find all the required filters and make sure the latest version of each is enabled. We can disable with Exceptions-U, if needed or desired.

I had all of the required filters ticked...I enabled block/modify JS methods.

That's the same Google URL Untangler filter that I have.

Those exceptions give me:

Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)

Which breaks Google again, fixes youtube freezing, but also gives the "you need adobe flash player" banner before videos start on YT - in other words, exactly what happens with the UA list updated.
Add Thank You Quote this message in a reply
Dec. 28, 2013, 04:02 PM
Post: #13
RE: User-Agents.ptxt update 2013-12-26
(Dec. 28, 2013 11:13 AM)slgar Wrote:  Can you please tell me how I use this information above to modify the user agent proxom sends to the server?

http://prxbx.com/forums/showthread.php?t...4#pid17394

HTH
Add Thank You Quote this message in a reply
Dec. 28, 2013, 04:11 PM
Post: #14
RE: User-Agents.ptxt update 2013-12-26
(Dec. 28, 2013 01:39 PM)Belarathon Wrote:  Which breaks Google again, fixes youtube freezing, but also gives the "you need adobe flash player" banner before videos start on YT - in other words, exactly what happens with the UA list updated.

I do see the "you need adobe flash player" banner before videos start on YT. That seems minor.

Could you attach a copy of dbug for a broken Google page. I want to see if the filter is just not matching.

Thanks
Add Thank You Quote this message in a reply
Dec. 28, 2013, 05:35 PM (This post was last modified: Dec. 28, 2013 05:40 PM by Belarathon.)
Post: #15
RE: User-Agents.ptxt update 2013-12-26
Here's what I get when I click on the first search return, as I did above:


Code:
HTTP/1.1 302 Redirect
Content-Length: 0
Connection: close
Location: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCsQFjAA&url=http%3A%2F%2Fwww.speedtest.net%2F&ei=j_6-UsTXCsnxoATbwoGIBQ&usg=AFQjCNHr6YyAkpiq37Q1AYAxF9ZW18CZrw&complete=0


Curiously, while most sites are reporting

Code:
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)

A few seem to detect the real user-agent,

Code:
GET / HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Proxy-Connection: Keep-Alive
Host: microsoft.com

In fact, if you just type "user-agent" in DuckDuckGo, it'll retrieve your true OS and UA.

Here's some reading on how IE11 UA differs from prior browsers...I thought this might be of help (or not). Not that I thought you needed it...this is just one of the sites that popped up when I googled "user-agent."

Understanding user-agent strings
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: