Post Reply 
SDCH = Shared Dictionary Compression over HTTP
May. 24, 2009, 11:19 PM
Post: #1
SDCH = Shared Dictionary Compression over HTTP
This is a technical note to mention SDCH and indicate why you might care about it. A friend came over today and was browsing through my proxy. I was surprised when the proxy listed occasional warnings about unknown encoding for some of his Google searches.

Most people here are familiar with a browser's cache and how it's used. Often a browser will test a server for new content using the "If-Modified-Since:" header. The server's response will either be a "304 Not Modified" if the content hasn't changed since the client's time, or "200 Ok" with the complete new content.

SDCH introduces a new wrinkle. It involves a server response that only contains the differences between two versions of the server's content. The client can indicate its support for SDCH by including mention in its "Accept-Encoding:" header that it sends to the server. When SDCH encoded content is sent by a server, then the client will use its previous copy and apply the given differences to that.

Problems are introduced when something like Proxo alters the server's content. When the client later tries to apply the SDCH differences, screwy results could be produced. The server has no way to know that its content had been manipulated. Its stated differences are only based on what the server knows about. Similarly, the client application doesn't know what Proxo may have manipulated. With hashing being done, hopefully the differences just don't get applied and you're stuck with the old version.

It might not be long before SDCH usage spreads further. The Google Toolbar has started using SDCH. Google's Chrome has support for it as well.

The following link shows example headers, and links to their white paper on the subject.
http://www.google.com/support/toolbar/bi...wer=113087

I think SDCH is likely to introduce future security issues. A filtering proxy, security appliance, or Anti-Virus application may see no problem with an initial content. Then the later differences could be almost useless when scanned on their own. It's only when you both parts together (or after more layered sets of differences) that the content will become malicious.

I believe Google's awareness of the potential for abuse is implied by their claim to exclude SDCH from SSL requests. You might not get nailed when logged in to your bank, but it's open season for everything else.

The apparent solution is to circumvent SDCH, either entirely always, or only for particular requests whose server content might be manipulated. The attribute for "sdch" would have to be removed from the client's "Accept-Encoding:" header, and related client headers like "Avail-Dictionary:" and "X-SDCH:" would have to be removed. Hopefully then the server won't send responses that are encoded with SDCH.

"Just Say No" to drugs and to drug-induced web encoding methods.
Add Thank You Quote this message in a reply
May. 24, 2009, 11:50 PM (This post was last modified: May. 24, 2009 11:57 PM by lnminente.)
Post: #2
RE: SDCH = Shared Dictionary Compression over HTTP
Thanks Graycode, here some filters to remove it, we still should need one filter more for the Avail-Dictionary headers

Code:
[HTTP headers]
In = FALSE
Out = TRUE
Key = "Accept-Encoding: Remove sdch {ln}090525 (out)"
Match = "\#(,$SET(tmp=.)|) sdch(,$SET(tmp=$GET(tmp).)($TST(tmp=..)$SET(#=,)|)|)( *)\#"
Replace = "\@"

In = FALSE
Out = TRUE
Key = "X-SDCH: Remove {ln}090525 (out)"
Match = "*"
Add Thank You Quote this message in a reply
May. 25, 2009, 04:55 PM
Post: #3
RE: SDCH = Shared Dictionary Compression over HTTP
Interesting!
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: