Cloudflare captcha [split] prox-config-sidki_2019-01-26b1

[ProxRocks]

Here's another (but this one is probably to be expected), so we now have three URLs for this behavior -

https://offerup.com/
https://raybuck.com/firebird-vs-trans-am...ry-trivia/
https://support.cloudflare.com/

Thanks in advance if you are able to find anything.

[JJoe]

So... We need the Proxomitron to share a browser's fingerprint.

blog.cloudflare.com/monsters-in-the-middleboxes/ Wrote:Introducing MITMEngine: Cloudflare’s HTTPS Interception Detector

Many TLS client implementations can be uniquely identified by features of the Client Hello message such as the supported version, cipher suites, extensions, elliptic curves, point formats, compression, and signature algorithms. The technique introduced by “The Security Impact of HTTPS Interception” is to construct TLS Client Hello signatures for common browser and middlebox implementations. Then, to identify HTTPS requests that have been intercepted, a server can look up the signature corresponding to the request’s HTTP User Agent, and check if the request’s Client Hello message matches the signature. A mismatch indicates either a spoofed User Agent or an intercepted HTTPS connection. The server can also compare the request’s Client Hello to those of known HTTPS interception tools to understand which interceptors are responsible for intercepting the traffic....
...
...To accomplish these goals, the Cryptography team at Cloudflare developed MITMEngine, an open-source HTTPS interception detector. MITMEngine is a Golang library that ingests User Agents and TLS Client Hello fingerprints, then returns the likelihood of HTTPS interception and the factors used to identify interception. To learn how to use MITMEngine, check out the project on GitHub.

MITMEngine works by comparing the values in an observed TLS Client Hello to a set of known browser Client Hellos. The fields compared include:

TLS version,
Cipher suites,
Extensions and their values,
Supported elliptic curve groups, and
Elliptic curve point formats.

When given a pair of User Agent and observed TLS Client Hello, MITMEngine detects differences between the given Client Hello and the one expected for the presented User Agent. For example, consider the following User Agent:

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/47.0.2526.111 Safari/537.36

This User Agent corresponds to Chrome 47 running on Windows 7. The paired TLS Client Hello includes the following cipher suites, displayed below as a hex dump:

0000 c0 2b c0 2f 00 9e c0 0a c0 14 00 39 c0 09 c0 13 .+./.... ...9....
0010 00 33 00 9c 00 35 00 2f 00 0a .3...5./ ..

These cipher suites translate to the following list (and order) of 13 ciphers:

TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)

The reference TLS Client Hello cipher suites for Chrome 47 are the following:

0000 c0 2b c0 2f 00 9e cc 14 cc 13 c0 0a c0 14 00 39 .+./.... .......9
0010 c0 09 c0 13 00 33 00 9c 00 35 00 2f 00 0a .....3.. .5./..

Looking closely, we see that the cipher suite list for the observed traffic is shorter than we expect for Chrome 47; two cipher suites have been removed, though the remaining cipher suites remain in the same order. The two missing cipher suites are

TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc14)
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc13)

Chrome prioritizes these two ChaCha ciphers above AES-CBC ciphers--a good choice, given that CBC (cipher block chaining) mode is vulnerable to padding oracle attacks. It looks like the traffic we received underwent HTTPS interception, and the interceptor potentially didn't support ChaCha ciphers.

Using contextual clues like the used cipher suites, as well as additional User Agent text, we can also detect which software was used to intercept the HTTPS connection. In this case, MITMEngine recognizes that the fingerprint observed actually matches a fingerprint collected from Sophos antivirus software, and indicates that this software is likely the cause of this interception.

We welcome contributions to MITMEngine. We are particularly interested in collecting more fingerprints of MITM software and browser TLS Client Hellos, because MITMEngine depends on these reference fingerprints to detect HTTPS interception. Contributing these fingerprints is as simple as opening Wireshark, capturing a pcap file with a TLS Client Hello, and submitting the pcap file in a PR. More instructions on how to contribute can be found in the MITMEngine documentation.

[ProxRocks]

Wow! Thanks!

My knowledge and insights were never as high as yours and sidki's - do you think we can "fake" a fingerprint that gets us past those cloudfare capcha's?

[JJoe]

(Feb. 04, 2022 04:46 PM)ProxRocks Wrote:  do you think we can "fake" a fingerprint that gets us past those cloudfare capcha's?

Maybe but it might not last long.

A couple years ago, I didn't think there was a way for me to get around it. I came to the conclusion that I had to actually remove the Proxomitron from the chain and my IP was probably flagged. Fortunately, I didn't use the sites that complained. Now, I'm wondering if I missed something. I was very not well.

So, this year, a site that I frequent started showing the captcha. This time, I quickly fixed an error in the user-agent header and the captcha went away. I could have missed that mistake before...

Now, more sites are protected or broken (depending on point of view) by this and people are looking for a solution. My searches have not found an acceptable working solution.

There is a game of whack-a-mole here but there are acceptable reasons to intercept https which may open a hole for 'unacceptable' access.

However, modifying the Proxomitron could be a lot of work with no guarantee of success, long term or short.

[JJoe]

I found https://httptoolkit.tech/blog/tls-fingerprinting-node-js/ interesting.

There must still be something wrong with me.

The... other side
https://github.com/cloudflare/mitmengine

Be careful with ja3er.com. You wouldn't want to give them a new fingerprint.

[amy]

Quote:However, modifying the Proxomitron could be a lot of work with no guarantee of success, long term or short.

Proxomitron Reborn can already specify the cipher configuration which OpenSSL will use. This is currently a static global option as is a little tricky to set it per-connection, but if the need arises, I will do it. If they are specifically looking for ciphersuites that OpenSSL doesn't have but browsers do (and hopefully they aren't ones that are actually going to be in use), that's going to be much harder to fix; but that doesn't mean I won't try either.

Sorry if opinion/commentary is not welcome here, but I absolutely HATE what the web has become, and in particular the browser racket. Cloudf**re is complicit in that. If we don't continue fighting against it, we may not be allowed to use anything but the "approved" browsers controlled by corporate and government interests.

"If we don't try, we have already lost."

[ProxRocks]

(Feb. 06, 2022 10:21 AM)amy Wrote:  ... I absolutely HATE what the web has become, and in particular the browser racket. Cloudfare is complicit in that. If we don't continue fighting against it, we may not be allowed to use anything but the "approved" browsers controlled by corporate and government interests.

"If we don't try, we have already lost."

Agreed!

I personally saw the writing on the wall way back in 2004!

"SSL" used to be a bank-only "technologoy", it is broken now that even malware sites can "buy/use" that "technology".

[Anno Domini]

....I 100% don't understand all the technical stuff you guys are talking about, I just wanted to post to say that you're awesome............thank you........

[rasczak]

(Feb. 06, 2022 10:21 AM)amy Wrote:  Proxomitron Reborn can already specify the cipher configuration which OpenSSL will use. This is currently a static global option as is a little tricky to set it per-connection, but if the need arises, I will do it. If they are specifically looking for ciphersuites that OpenSSL doesn't have but browsers do (and hopefully they aren't ones that are actually going to be in use), that's going to be much harder to fix; but that doesn't mean I won't try either.

Sorry if opinion/commentary is not welcome here, but I absolutely HATE what the web has become, and in particular the browser racket. Cloudf**re is complicit in that. If we don't continue fighting against it, we may not be allowed to use anything but the "approved" browsers controlled by corporate and government interests.

"If we don't try, we have already lost."

I agree with you Amy. Also, thank you again for the work you've already done to improve proxomitron.

Cloudflare is a problem for me because more than 2 businesses I interact with host their help pages using a separate domain (good for their security), but then put them behind cloudflare and configure it such that I get a 403 Forbidden + an hcaptcha for performing a single get request. I haven't figured out what is the root of the problem. I used to be able to use a proxy, such as https://en.wikipedia.org/wiki/Archive.today , but they now also show an hcaptcha.

[whenever]

(Feb. 06, 2022 10:21 AM)amy Wrote:  Proxomitron Reborn can already specify the cipher configuration which OpenSSL will use.

What if we just relay the Client Hello from the browser to the remote server (for example Cloudflare CDN) and make sure we don't touch the browser's User Agent string? Though I'm not sure if Proxomitron Reborn can do this kind of low level stuff.

[whenever]

(Feb. 05, 2022 05:35 PM)JJoe Wrote:  Now, more sites are protected or broken (depending on point of view) by this and people are looking for a solution. My searches have not found an acceptable working solution.

What if we provide an unknown user agent say "Chrome 222" so there isn't a match in the fingerprints database and they will let us pass? Maybe we also need to randomize the user agent because our request may lead to a record to be created in the database later.

[JJoe]

(Feb. 27, 2022 03:38 AM)whenever Wrote:  What if we provide an unknown user agent say "Chrome 222" so there isn't a match in the fingerprints database and they will let us pass? Maybe we also need to randomize the user agent because our request may lead to a record to be created in the database later.

It didn't work for me. But then, I didn't try SpaceBison.
It may be that some Cloudflare instances deny access to unusual user agents.

[amy]

I can say from experience - trying to write a browser (long story...) - that having an "unusual" UA header, or even none at all, definitely sticks out like the proverbial sore thumb and gets you blocked from a lot of sites even beyond Cloudflare. The classic SpaceBison has already somehow become famous enough to be blacklisted by many "bot detectors".

(Feb. 26, 2022 03:50 AM)whenever Wrote:  

(Feb. 06, 2022 10:21 AM)amy Wrote:  Proxomitron Reborn can already specify the cipher configuration which OpenSSL will use.

What if we just relay the Client Hello from the browser to the remote server (for example Cloudflare CDN) and make sure we don't touch the browser's User Agent string? Though I'm not sure if Proxomitron Reborn can do this kind of low level stuff.

The ClientHello sets up the SSL/TLS handshake negotiation, so it could theoretically specify stuff that Proxo's OpenSSL doesn't know how to handle, like some ciphers, if the server decides to pick one.

This is an issue that has been on my mind for a while and unfortunately I currently don't have much time to work on it, but others have also been doing some work on beating TLS handshake fingerprinting:

Code:

https://github.com/lwthiker/curl-impersonate

[cattleyavns]

mitmproxy devs are trying to figure out how to deal with TLS Fingerprint, I think they had some ideas about handling User-Agent: https://github.com/mitmproxy/mitmproxy/issues/4575

If I understand correctly, just make a pair of Cipher and User-Agent.

Related Articles:
- https://httptoolkit.tech/blog/tls-finger...g-node-js/
- https://hacker-news.news/post/30378562 (Comment section is pretty interesting)

[cattleyavns]

I've managed to beat TLS Fingerprint with python urllib3 + override DEFAULT_CIPHERS in Lib\site-packages\urllib3\util\_ssl.py with Pale Moon/any browser's ciphersuite and change User-Agent to match Pale Moon/any browser. And the final result, my transparent local proxy can now bypass CloudFlare's TLS Fingerprint!

Should I share the whole progress ? I don't know, I just afraid CloudFlare will just patch this exploit (and it's very easy to block because OpenSSL always send TLS_EMPTY_RENEGOTIATION_INFO_SCSV, and real Pale Moon/web browser doesn't, so they can easily block my fingerprint by checking User-Agent and TLS_EMPTY_RENEGOTIATION_INFO_SCSV, if it's Pale Moon+TLS_EMPTY_RENEGOTIATION_INFO_SCSV then block). Basically I did:

- I used Wireshark to capture real Pale Moon's ciphersuite in Client Hello packet (Pale Moon is my main browser), then converted Pale Moon's ciphersuite format to Python's ciphersuite format, override DEFAULT_CIPHERS in Lib\site-packages\urllib3\util\_ssl.py with my new ciphersuite.

This is my video (tested on alternativeto.net) https://streamable.com/btr8f8