Privoxy with ProxHTTPSProxyMII v1.3a on linux

[rogern]

I was installing pip3 also as i can recall beside easy_install for python
Do keep in mind that i am on a raspberryp running debian
But i do have

[email protected]:/usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a# python3.4
Python 3.4.3+ (default, Oct 10 2015, 09:15:38)
[GCC 5.2.1 20151003] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

I had to run apt-get update ; apt-get upgrade -f -y --force-yes

when installing with pip now i get

pip install pyopenssl
Traceback (most recent call last):
File "/usr/local/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3080, in <module>
@_call_aside
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3066, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3093, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 651, in _build_master
ws.require(__requires__)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 952, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 839, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==7.1.2' distribution was not found and is required by the application

pip3.4 installs the modules but get this error still it is installed

[email protected]:/usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a# ./ProxHTTPSProxy.py
Traceback (most recent call last):
File "./ProxHTTPSProxy.py", line 20, in <module>
import urllib3
ImportError: No module named 'urllib3'

Here you see i have installed them

[email protected]:/usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a# pip3.4 install pyopenssl
Downloading/unpacking pyopenssl
Downloading pyOpenSSL-0.15.1-py2.py3-none-any.whl (102kB): 102kB downloaded
Downloading/unpacking six>=1.5.2 (from pyopenssl)
Downloading six-1.10.0-py2.py3-none-any.whl
Downloading/unpacking cryptography>=0.7 (from pyopenssl)
Downloading cryptography-1.0.2.tar.gz (332kB): 332kB downloaded
Running setup.py (path:/tmp/pip_build_root/cryptography/setup.py) egg_info for package cryptography

no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*' found under directory 'vectors'
Downloading/unpacking idna>=2.0 (from cryptography>=0.7->pyopenssl)
Downloading idna-2.0-py2.py3-none-any.whl (61kB): 61kB downloaded
Downloading/unpacking pyasn1>=0.1.8 (from cryptography>=0.7->pyopenssl)
Downloading pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python3.4/site-packages (from cryptography>=0.7->pyopenssl)
Downloading/unpacking cffi>=1.1.0 (from cryptography>=0.7->pyopenssl)
Downloading cffi-1.3.0.tar.gz (347kB): 347kB downloaded
Running setup.py (path:/tmp/pip_build_root/cffi/setup.py) egg_info for package cffi

Downloading/unpacking pycparser (from cffi>=1.1.0->cryptography>=0.7->pyopenssl)
Downloading pycparser-2.14.tar.gz (223kB): 223kB downloaded
Running setup.py (path:/tmp/pip_build_root/pycparser/setup.py) egg_info for package pycparser

warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
Installing collected packages: pyopenssl, six, cryptography, idna, pyasn1, cffi, pycparser
Running setup.py install for cryptography

Installed /tmp/pip_build_root/cryptography/cffi-1.3.0-py3.4-linux-armv7l.egg
Searching for pycparser
Reading https://pypi.python.org/simple/pycparser/
Best match: pycparser 2.14
Downloading https://pypi.python.org/packages/source/...4b51a4f935
Processing pycparser-2.14.tar.gz
Writing /tmp/easy_install-9um5iv48/pycparser-2.14/setup.cfg
Running pycparser-2.14/setup.py -q bdist_egg --dist-dir /tmp/easy_install-9um5iv48/pycparser-2.14/egg-dist-tmp-q8vavdlv
warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
zip_safe flag not set; analyzing archive contents...

Installed /tmp/pip_build_root/cryptography/pycparser-2.14-py3.4.egg

no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*' found under directory 'vectors'
generating cffi module 'build/temp.linux-armv7l-3.4/_padding.c'
generating cffi module 'build/temp.linux-armv7l-3.4/_constant_time.c'
generating cffi module 'build/temp.linux-armv7l-3.4/_openssl.c'
building '_openssl' extension
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.4m -c build/temp.linux-armv7l-3.4/_openssl.c -o build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_openssl.o
gcc -pthread -shared build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_openssl.o -lssl -lcrypto -o build/lib.linux-armv7l-3.4/cryptography/hazmat/bindings/_openssl.cpython-34m.so
building '_constant_time' extension
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.4m -c build/temp.linux-armv7l-3.4/_constant_time.c -o build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_constant_time.o
gcc -pthread -shared build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_constant_time.o -o build/lib.linux-armv7l-3.4/cryptography/hazmat/bindings/_constant_time.cpython-34m.so
building '_padding' extension
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python3.4m -c build/temp.linux-armv7l-3.4/_padding.c -o build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_padding.o
gcc -pthread -shared build/temp.linux-armv7l-3.4/build/temp.linux-armv7l-3.4/_padding.o -o build/lib.linux-armv7l-3.4/cryptography/hazmat/bindings/_padding.cpython-34m.so
Running setup.py install for cffi
building '_cffi_backend' extension
gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DUSE__THREAD -I/usr/local/include/python3.4m -c c/_cffi_backend.c -o build/temp.linux-armv7l-3.4/c/_cffi_backend.o
gcc -pthread -shared build/temp.linux-armv7l-3.4/c/_cffi_backend.o -lffi -o build/lib.linux-armv7l-3.4/_cffi_backend.cpython-34m.so

Running setup.py install for pycparser

warning: no previously-included files matching 'yacctab.*' found under directory 'tests'
warning: no previously-included files matching 'lextab.*' found under directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under directory 'examples'
warning: no previously-included files matching 'lextab.*' found under directory 'examples'
Build the lexing/parsing tables
Successfully installed pyopenssl six cryptography idna pyasn1 cffi pycparser
Cleaning up...
[email protected]:/usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a# pip3.4 install urllib3
Downloading/unpacking urllib3
Downloading urllib3-1.12-py2.py3-none-any.whl (85kB): 85kB downloaded
Installing collected packages: urllib3
Successfully installed urllib3
Cleaning up...
[email protected]:/usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a# pip3.4 install colorama
Downloading/unpacking colorama
Downloading colorama-0.3.3.tar.gz
Running setup.py (path:/tmp/pip_build_root/colorama/setup.py) egg_info for package colorama

Installing collected packages: colorama
Running setup.py install for colorama

Successfully installed colorama
Cleaning up...

[whenever]

It seems you have multiple version of Python installed. What if you run

Code:

python3.4 /usr/local/lib/python3.4/ProxHTTPSProxyMII_py-1.3a/ProxHTTPSProxy.py

BTW, you don't need to put ProxHTTPSProxy that deep. You can put it anywhere where you can easily find it.

[kik0s]

you need to upgrade you proxhttps to the newest versions. it supports the newest urllib. then it should work.

[rogern]

Have to say it is working atleast it is starting, the fault was i was starting from the wrong folder

[email protected]:~/https-proxy/ProxHTTPSProxyMII_py-1.3# python3.4 ProxHTTPSProxy.py
============================================================================
ProxHTTPSProxyMII v1.3.1 (urllib3/1.9.1)

FrontServer : localhost:8079
RearServer : localhost:8081
ParentServer : http://192.168.2.1:8118
Proxomitron : http://localhost:8080
============================================================================


Have imported the CA.cert to the browser, privoxy runs on actually 127.0.0.1:8118 but 192.168.1.1 and 192.168.2.1 is the gateway so i have 2 dietpie`s on different subnets.

But how do you see it actually works by blocking https sites as i do not have privoxy on localhost?

user.action is like this

{ +forward-override{forward 192.168.1.200:8081} } = dietpi green network
TAG:.*?ProxHTTPSProxyMII
#https proxy
{ +client-header-tagger{tagger4https} }
/
{ +forward-override{forward 192.168.2.61:8081} } = dietpi purple network
TAG:.*?ProxHTTPSProxyMII

config.ini

[GENERAL] dietpi purple
ProxAddr = http://localhost:8080
FrontPort = 8079
RearPort = 8081
DefaultProxy = http://192.168.2.1:8118

[rogern]

Got it working

[v4victory]

I'd like to know how do you add Privoxy and ProxHTTPSProxy to startup?
Which command?

[kik0s]

(Apr. 23, 2016 03:10 AM)v4victory Wrote:  I'd like to know how do you add Privoxy and ProxHTTPSProxy to startup?
Which command?

i have created an init script. will post the code later.

[vlad_s]

Hello! Sorry for my English (communicate through an online translator). After the launch, I see the message "ImportError":
Traceback (most recent call last):
File "ProxHTTPSProxy.py", line 25, in <module>
from import ThreadingMixIn socketserver
ImportError: No module named socketserver
Pip install socketserver type command, does not give a positive result.
What is socketserver and where can I get it?
Output pip list:
CFFI (1.7.0)
chardet (2.3.0)
Colorama (0.3.7)
configparser (3.5.0)
Cryptography (1.5)
defusedxml (0.4.1)
dnspython (1.12.0)
docopt (0.6.2)
docutils (0.12)
enum34 (1.1.6)
html5lib (0.999)
httpserver (1.1.0)
IDNA (2.1)
IPAddress (1.0.16)
NDG-httpsclient (0.3.2)
Pillow (2.6.1)
PIP (8.1.2)
ply (3.4)
pyasn1 (0.1.9)
pycparser (2.14)
pycrypto (2.6.1)
Pygments (2.0.1)
pyOpenSSL (16.1.0)
PySocks (1.5.7)
Python-apt (0.9.3.12)
Python-debian (0.1.27)
Python-debianbts (1.11)
reportbug (6.6.3)
requests (2.4.3)
Roman (2.0.0)
setuptools installed (26.1.1)
six (1.8.0)
Soappy (0.12.22)
urllib3 (1.16)
wheel (0.29.0)
wstools (0.4.3)

Debian OS 8.5, amd64.
Thank you!

[JJoe]

(Sep. 06, 2016 07:40 PM)vlad_s Wrote:  ImportError: No module named socketserver

Hi,
Python 3 uses socketserver.
Python 2 uses SocketServer.

Are you using Python 3?

[vlad_s]

Hi, Thank you for your response! Run the command python /path/ProxHTTPSProxy.py. This python 2 as I understand? Tried python3 /path/ProxHTTPSProxy.py, see another error:

[email protected]:/ProxHTTPSProxyMII# python3 ProxHTTPSProxy.py
Traceback (most recent call last):
File "ProxHTTPSProxy.py", line 21, in <module>
from urllib3.contrib.socks import SOCKSProxyManager
ImportError: No module named 'urllib3.contrib.socks'

pip3 list:
cffi (1.7.0)
chardet (2.3.0)
colorama (0.3.2)
cryptography (1.5)
docopt (0.6.2)
html5lib (0.999)
httpserver (1.1.0)
idna (2.1)
micropython-socketserver (0.0.0)
pip (1.5.6)
pyasn1 (0.1.9)
pycparser (2.14)
pyOpenSSL (16.1.0)
PySocks (1.5.7)
requests (2.4.3)
setuptools (26.1.1)
six (1.8.0)
urllib3 (1.9.1)
wheel (0.24.0)

Again something is missing?

It seems to be working. Returned to pure image Debian OS and did the following:
apt-get install build-essential libssl-dev libffi-dev python3-dev
pip3 install colorama
pip3 install urllib3
pip3 install pyOpenSSL
pip3 install PySocks (as in neighbouring topic about OS X Tutorial)

Launched python3 ProxHTTPSProxy.py and saw:
============================================================================
ProxHTTPSProxyMII v1.4 (urllib3/1.17)

FrontServer: localhost: 8079
RearServer: localhost: 8081
ParentServer: None
Proxomitron: http://localhost:8080
============================================================================

So I figured that the launch was a success?

[JJoe]

(Sep. 08, 2016 04:25 PM)vlad_s Wrote:  ImportError: No module named 'urllib3.contrib.socks'
...
Again something is missing?

Socks support was added to urllib3 in version 1.14 (2015-12-29)

(Sep. 08, 2016 04:25 PM)vlad_s Wrote:  pip3 install urllib3

May have updated urllib3 to a version with socks support.

(Sep. 08, 2016 04:25 PM)vlad_s Wrote:  So I figured that the launch was a success?

Well... I haven't used it with debian. That is the expected header.
Does it work?

[vlad_s]

Apparently still works . But the problem appeared after a while (less than a minute) stop open sites. Have to kill Pythons and run again. Then everything repeats itself.
When running, see log in the Terminal, when an issue log is not written as if no requests to the proxy and browser when this long thinks and gives that "cannot display this page". Experiments spending Oracle VM VirtualBox

pip3 list
cffi (1.8.2)
colorama (0.3.7)
cryptography (1.5)
idna (2.1)
pip (8.1.2)
pyasn1 (0.1.9)
pycparser (2.14)
pyOpenSSL (16.1.0)
PySocks (1.5.7)
setuptools (26.1.1)
six (1.10.0)
urllib3 (1.14)
wheel (0.29.0)

[JJoe]

I have not used ProxHTTPSProxyMII with VirtualBox.

(Sep. 09, 2016 06:00 PM)vlad_s Wrote:  But the problem appeared after a while (less than a minute) stop open sites. Have to kill Pythons and run again. Then everything repeats itself.

Does it work without privoxy?
Change config.ini to test.

Code:

[GENERAL]
ProxAddr = http://localhost:8081
FrontPort = 8079
RearPort = 8081

Restart ProxHTTPSProxyMII after editing config.ini proxy settings.

(Sep. 09, 2016 06:00 PM)vlad_s Wrote:  urllib3 (1.14)

Use current version urllib3. It should have socks module.

[vlad_s]

Without privoxy? Don't know how to start. Guided by this line: https://www.prxbx.com/forums/showthread.php?tid=2224
Part of the config.ini (I changed):
[GENERAL]
ProxAddr = http://localhost:8118
FrontPort = 8079
RearPort = 8081

By the way, today launched the real machine (Debian OS 8.5 as on VirtualBox) with installed privoxy + squid configured and working, the behavior is the same, just a little bit going and "it was not possible to open this page," in general the same. urllib3 latest version. If you beat a Python, the error in the browser is another, "the proxy server is not responding".

[JJoe]

(Sep. 10, 2016 09:58 PM)vlad_s Wrote:  Guided by this line: https://www.prxbx.com/forums/showthread.php?tid=2224

https://www.prxbx.com/forums/showthread.php?tid=2224 is for OS X...

I see https://sourceforge.net/p/ijbswa/support...#c213/77fd

Cattleya aka cattleyavns Wrote:You should install urllib3 1.9.1 only because new version will have problem with headers so ProxHTTPSProxy will fail to load https sites.

also https://sourceforge.net/p/ijbswa/support...1667/#3f7d

korda aka Faxopita Wrote:Updating to the latest version of ProxHTTPSProxy: ProxHTTPSProxyMII v1.3a
First, obtain both archive: ProxHTTPSProxyMII_py 1.3.zip and ProxHTTPSProxyMII_py 1.3a.zip
Copy the content of ProxHTTPSProxyMII_py 1.3a folder over its corresponding—existing—Python scripts in ProxHTTPSProxyMII_py 1.3 folder.
Uninstall previous version of urllib3:
pip3 uninstall urllib3
Install urllib3/1.10.2:
pip3 install urllib3==1.10.2
Replace previous [email protected]: Proxomitron FrontProxy@$[email protected] (cf. user.filter) by new one: s@^.*Tagged:.*ProxHTTPSProxyMII.*FrontProxy.*@$[email protected]
Replace previous TAG:.*?Proxomitron (cf. user.action) by new one: TAG:.*?ProxHTTPSProxyMII
You're done!

You may want to start with ProxHTTPSProxyMII_py 1.3a and an older version of urllib3.
This would avoid socks module error and may answer some questions.

(Sep. 10, 2016 09:58 PM)vlad_s Wrote:  Without privoxy? Don't know how to start. Guided by this line: https://www.prxbx.com/forums/showthread.php?tid=2224
Part of the config.ini (I changed):
[GENERAL]
ProxAddr = http://localhost:8118
FrontPort = 8079
RearPort = 8081

You have
front server at 8079 >> privoxy at 8118 >> rear server at 8081 >> internet

Code:

[GENERAL]
ProxAddr = http://localhost:8081
FrontPort = 8079
RearPort = 8081

Would be
front server at 8079 >> rear server at 8081 >> internet
without privoxy.