2019-09-12 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: SK NZ <sa***m@gmail.com>
Message Hash: 5974f768d06503a5c344c68f02362320e7a80dbb529bdf5c1d398c67fc67c7e5
Message ID: <08020c3b-e607-46fc-9f3b-0cc50e26bb41@grasehotspot.org>
Reply To: <CAESLx0LcePJj_V-kGgrkiYOQdpZndZG9zYb2-mFZ-COp8hXSjA@mail.gmail.com>
UTC Datetime: 2019-09-12 06:49:47 UTC
Raw Date: Thu, 12 Sep 2019 06:49:47 -0700

Raw message 

Hello Tim,
I'm stuck with this issue for the last 3 days. If you kindly give a look at 
it that will be really life-saving! I've added port *3127* in the firewall 
rules as you suggested but no luck yet! *I cannot browse HTTP or HTTPS 
sites under AP hotspot*. It's a fresh grase hotspot server, minor 
modifications are listed below.

*iptables, squid, chili rules:*
 https://paste.grasehotspot.org/view/raw/80fcc90f

After a fresh boot, I can test squid from ssh(*squidclient  
https://grasehotspot.org*). It works until 1st user login using AP hotspot, 
after that squidclient also stops responding. I've checked squid3 status, 
it says *active(running)*. So I guess iptables or chilli is doing something 
to squid, that's weird!

Exact same squid configuration works in the same version of squid and OS in 
a standalone server, but it's not working here. I believe the only 
difference is iptable/nat rules. *Another worthy thing to mention,* after a 
fresh grase hotspot server installation it doesn't work out of the box. For 
default squid.conf.grase, it throws *Error: No forward-proxy ports 
configure. *So I have to add a forward port.

http_port 3128
> http_port 3127 intercept


ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 80 -j REDIRECT --to 3127


I'm using stable *Squid 3.4.8 @ Debian Jessie*, I guess from this version 
it's a must to specify a forward port. This stops throwing an error in 
squid cache log, still, HTTP sites are not working. It shows *ERROR/Invalid 
URL* squid error page. So I have to modify *squid.conf.grase* again to make 
it work!

http_port 3128 accel vhost allow-direct


It's not a transparent proxy anymore, rather a reverse proxy. Now HTTP 
sites are working and also monitor session works in *grase/radmin*. *So 
INTERCEPT is not working for only HTTP or both.*

intercept means take the origin server details from the NAT system.
> vs.
> accel means Squid is providing CDN services for the domain being fetched.



I guess something wrong with NAT here when we use one port for intercept 
another for forwarding? I've done a lot of digging last 3 days, you may 
look at this( 
squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Transparent-intercept-Issues-td4681851.html 
). I'm not sure this is relevant for my issue or not.

*Thanks in advance.*

 









On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>
> It looks like you might need to add a firewall rule to allow 3127 to the 
> Grase server.
>
> However, without installing CA certificates on the client, what benefit do 
> you see from "proxying" HTTPS connections through squid?
>
> My long term plan was to remove Squid from future versions of the hotspot, 
> because the logs are becoming useless due to HTTPS traffic. I believe the 
> only thing you can get is the hostname it's connecting to, and even that 
> may not work with HTTPS 2.
>
> Regards
>
> Tim
>
> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com <javascript:>> 
> wrote:
>
>> Hello,
>> I've managed to compile squid3 with SSL support in a standalone(squid3 
>> only) server, now I can monitor HTTPS traffic log without issuing any 
>> certificate. I got this idea originally from here: 
>> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>>
>> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
>> packages with SSL support, also kept all original Grase configurations. Now 
>> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
>> perfectly, squid restarted without any error.
>>
>> http_port 3128
>>> http_port 3129 intercept
>>> https_port 3127 intercept ssl-bump generate-host-certificates=off 
>>> cert=/etc/squid3/certs/squid.pem
>>> acl ssl-bump_port myportname 3127
>>> always_direct allow ssl-bump_port
>>
>>
>> For this new squid ports, default IPTABLE rules will not work. So I tried 
>> to modify* /etc/chilli/ipub.sh*
>>
>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 -j 
>>> DROP
>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>> --dport 80 -j REDIRECT --to 3129
>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 
>>> -j DROP
>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>> --dport 443 -j REDIRECT --to 3127
>>> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>>
>>
>> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it 
>> somewhere!. If anyone can help to figure it out, that will be really a 
>> great-great help.
>>
>>
>> In my standalone server, I've used this IPTABLE rules and it works like a 
>> charm!
>>
>> *nat
>>> :PREROUTING ACCEPT
>>> :POSTROUTING ACCEPT
>>> :OUTPUT ACCEPT
>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>> --dport 443 -j REDIRECT --to-ports 3127
>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>> --dport 80 -j REDIRECT --to-ports 3129
>>> COMMIT
>>> *filter
>>> :INPUT DROP
>>> :FORWARD DROP
>>> :OUTPUT ACCEPT
>>> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>> -A INPUT -i lo -j ACCEPT
>>> -A OUTPUT -o lo -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
>>> ACCEPT 
>>> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
>>> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
>>> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
>>> COMMIT
>>
>>
>>
>> *Thanks in advance.*
>>
>>
>>  
>>
>> -- 
>> This mailing list is for the Grase Hotspot Project 
>> http://grasehotspot.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Grase Hotspot" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to gr***.@grasehotspot.org <javascript:>.
>> To view this discussion on the web visit 
>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
>> .
>>
>

Thread