Post Reply 
Gmail Technical Rant
Feb. 25, 2009, 04:26 PM
Post: #16
RE: Gmail Technical Rant
(Feb. 25, 2009 04:05 PM)lnminente Wrote:  For one month i'm blocking the requests wich produces the 204 response from google and new banners from youtube.

Yep, i've even seen that via (now blocked) prefetch link tags at Google searches (but very rarely).

Code:
http://o-o.resolver.o.89.247.103.118.896b25a7a45d5f84.l.google.com/gen_204

("89.247.103.118" would be my IP back then.)
Add Thank You Quote this message in a reply
Feb. 25, 2009, 04:32 PM
Post: #17
RE: Gmail Technical Rant
Now, here is an alternative approach.
Probably less intrusive, but appears to me rather adventurous.
To my surprise, it works!

Code:
[HTTP headers]
In = TRUE
Out = FALSE
Key = "Content-Length: 2 204 - Add if Missing    9.02.25 (In) TEST"
URL = "$RESP(204)"
Match = "^?"
Replace = "0"

I could imagine that it stands and falls with possible 204 content never being parsed, always being ignored, by all user-agents.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 04:54 PM (This post was last modified: Feb. 25, 2009 05:16 PM by Graycode.)
Post: #18
RE: Gmail Technical Rant
NOTE: I see now there's been other thread responses since I started typing this, so it may appear out-of-line. Oh well, here it is anyway Wink

(Feb. 25, 2009 10:42 AM)sidki3003 Wrote:  No, constructing an incoming "Connection: close" header doesn't work.

What's the chances that Scott might have overlooked "Proxy-Connection: close" ? It could have the same effect if Proxo allows you to send that to the browser.

Quote:Could the content of a 204 response ever (or theoretically) be parsed by the browser?

RFC 2616 states: "All 1xx (informational), 204 (no content), and 304 (not modified) responses MUST NOT include a message-body." However, the section specific to 204 responses also states: "The response MAY include new or updated metainformation in the form of entity-headers, which if present SHOULD be associated with the requested variant."

So Yes, the headers of a 204 could be important. Google may be using it in the problem URL just to assign the cookie value.

Other scenarios like WebDAV may be using its headers too. For example: http://msdn.microsoft.com/en-us/library/aa565115.aspx
"With PUT, the 204 response allows the server to send back an updated etag and other entity information about the resource that has been affected by the PUT operation. This allows the client to do the next PUT using the If-Match precondition to ensure that edits are not lost."

Quote:Should i modify the content-length test in the filter to only redirect if the Content-Length header is missing?

That should be fine either way. A value other than zero on a 204 is unlikely and would be non-compliant. What seems to be triggering the bug is Proxo not assuming zero content without the Length header yet the browsers do. When there is a Content-Length then Proxo shouldn't have a problem.
(Feb. 25, 2009 04:32 PM)sidki3003 Wrote:  Now, here is an alternative approach.
Probably less intrusive, but appears to me rather adventurous.
To my surprise, it works!
Sorry I don't understand, but the "it works" part sounds wonderful! Is it telling Proxo that the Content-Length is zero so that Proxo then knows not to expect anything more for the 204? If so, I'm surprised Proxo wouldn't let you tell the browser to close the connection but it will let you tell Proxo the length.

Quote:I could imagine that it stands and falls with possible 204 content never being parsed, always being ignored, by all user-agents.
The length will not be ignored by user-agents, but it may be validated as being zero since no payload is permitted.

For the 1xx, 204, and 304 responses, my proxy validates any length given (helping to insure no data payload), and then always gives "Content-Length: 0" back to the browsers as additional clarity. If the server tries to sneak something by sending anything after the headers then the proxy terminates its connection to that bogus server.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 05:35 PM
Post: #19
RE: Gmail Technical Rant
(Feb. 25, 2009 04:54 PM)Graycode Wrote:  What's the chances that Scott might have overlooked "Proxy-Connection: close" ? It could have the same effect if Proxo allows you to send that to the browser.

Wow! That one is indeed sent to the browser. Thanks Graycode!

Adding an incoming "Proxy-Connection:close" closes the connection for the browser (but not for Proxomitron, see below). "Garbage" problem solved.


Quote:Sorry I don't understand, but the "it works" part sounds wonderful! Is it telling Proxo that the Content-Length is zero so that Proxo then knows not to expect anything more for the 204? If so, I'm surprised Proxo wouldn't let you tell the browser to close the connection but it will let you tell Proxo the length.

Yep! That's apparently what's happening. (edit: It sounds really odd, so i've re-tested. Yes, 204 connection gets closed immediately.)

Adding a missing incoming "Content-Length:0" closes the connection for both, the browser and Proxomitron. "Garbage" problem solved, too.


Quote:For the 1xx, 204, and 304 responses, my proxy validates any length given (helping to insure no data payload), and then always gives "Content-Length: 0" back to the browsers as additional clarity. If the server tries to sneak something by sending anything after the headers then the proxy terminates its connection to that bogus server.

If i got you right, it would mean that the last approach, "Content-Length:0", which i'd prefer b/c it also closes Prox' connection, is safe.

*And* that i could also exclude 204s from being web-filtered (i.e. ignoring a possible message body). That's regarding the "always filter, if no content-type" strategy, i've mentioned earlier.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 06:49 PM
Post: #20
RE: Gmail Technical Rant
(Feb. 25, 2009 05:35 PM)sidki3003 Wrote:  If i got you right, it would mean that the last approach, "Content-Length:0", which i'd prefer b/c it also closes Prox' connection, is safe.

*And* that i could also exclude 204s from being web-filtered (i.e. ignoring a possible message body). That's regarding the "always filter, if no content-type" strategy, i've mentioned earlier.

Yup, you got it right. Specifying Length:0 on any 1xx, 204, or 304 response should be safe. Of those, the 304 is extremely common.

As for being web-filtered, since these "MUST NOT" have a message body then filtering shouldn't be applicable. Short of the stupid / ignorant part of the internet, all user-agents will work with your Length:0 solution. It's a common standards-compliant thing to do.

I do occasionally run into non-compliant server responses. Often they are "kids" using home-grown software spitting out things like their avatars from forums that allow remote references for images. If a non-compliant server tries to give those any body content, that could just as easily cause problems for browsers regardless of whether Proxo is in the mix.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 07:24 PM
Post: #21
RE: Gmail Technical Rant
Okay. Thanks a lot for helping and the inspiring discussion. Smile!
I've posted the filter here:
http://prxbx.com/forums/showthread.php?tid=1292

And sorry about having hijacked this thread!

edit: After posting, i've extended my own test version to also match on 304s (as well as 1xx, although possibly not relevant). And i've set up logging for all such replies with a Content-Length value greater than 0.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 09:09 PM
Post: #22
RE: Gmail Technical Rant
(Feb. 25, 2009 07:24 PM)sidki3003 Wrote:  And sorry about having hijacked this thread!

You turned a Rant thread into something useful. Smile!
Anyway, the additional discussion might draw in a Google programmer who will spot my whining and fix their server-side delay.

Quote:edit: After posting, i've extended my own test version to also match on 304s (as well as 1xx, although possibly not relevant). And i've set up logging for all such replies with a Content-Length value greater than 0.

In the new thread I saw:
Code:
URL = "^$RESP(204|3)| ...
Forgive my Proxo ignorance again, but should that be (204|304} ? If not, what's the 3 for?

Also: I have no idea how Proxo handles the "100 Continue" response, but surely it has special consideration for it. The 100 is an intermediate set of headers that will be later followed by the real set of response headers and data. So the 100 is not an "answer", it's notfication that the answer will be provided soon. That happens a lot when POST to many IIS servers. I'd be surprised if Proxo even lets your filters see or know about the 100 for possible logging. The rest of the 1xx series are extremely rare.
Add Thank You Quote this message in a reply
Feb. 25, 2009, 09:28 PM
Post: #23
RE: Gmail Technical Rant
(Feb. 25, 2009 09:09 PM)Graycode Wrote:  Forgive my Proxo ignorance again, but should that be (204|304} ? If not, what's the 3 for?

$RESP(3) matches all 3xx responses. It basically says: trust them if without content-type or a text/plain one (some servers are configured to set text/plain, even for redirects, if no other value is given).


As for 1xx, ahh okay, i'll let you know if i get any matches.
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: