2019-09-12 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: SK NZ <sa***m@gmail.com>
Message Hash: 8a8472ffd868fbdbcf6857347fdf9e828577ad3aedf31a5fba5459edea5a4782
Message ID: <c91a341d-7b47-4238-b8d8-1ed694cfc031@grasehotspot.org>
Reply To: <b5f30b5c-0e52-42a2-bd57-d1f2df78ab3c@grasehotspot.org>
UTC Datetime: 2019-09-12 10:40:18 UTC
Raw Date: Thu, 12 Sep 2019 10:40:18 -0700

Raw message 

Hello Christopher,
Good to see you again as a savior! I'm a bit confused with the provided 
link's solution. Can you please help me to adjust with my current rules?

sudo nano /etc/chilli/ipup.sh 

ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3126 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 80 -j REDIRECT --to 3126 

ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 443 -j REDIRECT --to 3127 

ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE



Thanks again.


On Thursday, September 12, 2019 at 10:30:31 PM UTC+6, christopher wrote:
>
> Hello,
>
> I just did a quick look on the net, and the following may give you the 
> solution:
>
> https://serverfault.com/questions/870319/ip-tables-intercept-exception
>
> Regards,
>
> Christopher.
>
> On Friday, 13 September 2019 01:49:48 UTC+12, SK NZ wrote:
>>
>> Hello Tim,
>> I'm stuck with this issue for the last 3 days. If you kindly give a look 
>> at it that will be really life-saving! I've added port *3127* in the 
>> firewall rules as you suggested but no luck yet! *I cannot browse HTTP 
>> or HTTPS sites under AP hotspot*. It's a fresh grase hotspot server, 
>> minor modifications are listed below.
>>
>> *iptables, squid, chili rules:* 
>> https://paste.grasehotspot.org/view/raw/80fcc90f
>>
>> After a fresh boot, I can test squid from ssh(*squidclient  
>> https://grasehotspot.org <https://grasehotspot.org>*). It works until 
>> 1st user login using AP hotspot, after that squidclient also stops 
>> responding. I've checked squid3 status, it says *active(running)*. So I 
>> guess iptables or chilli is doing something to squid, that's weird!
>>
>> Exact same squid configuration works in the same version of squid and OS 
>> in a standalone server, but it's not working here. I believe the only 
>> difference is iptable/nat rules. *Another worthy thing to mention,* 
>> after a fresh grase hotspot server installation it doesn't work out of the 
>> box. For default squid.conf.grase, it throws *Error: No forward-proxy 
>> ports configure. *So I have to add a forward port.
>>
>> http_port 3128
>>> http_port 3127 intercept
>>
>>
>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
>>> DROP
>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>> --dport 80 -j REDIRECT --to 3127
>>
>>
>> I'm using stable *Squid 3.4.8 @ Debian Jessie*, I guess from this 
>> version it's a must to specify a forward port. This stops throwing an error 
>> in squid cache log, still, HTTP sites are not working. It shows *ERROR/Invalid 
>> URL* squid error page. So I have to modify *squid.conf.grase* again to 
>> make it work!
>>
>> http_port 3128 accel vhost allow-direct
>>
>>
>> It's not a transparent proxy anymore, rather a reverse proxy. Now HTTP 
>> sites are working and also monitor session works in *grase/radmin*. *So 
>> INTERCEPT is not working for only HTTP or both.*
>>
>> intercept means take the origin server details from the NAT system.
>>> vs.
>>> accel means Squid is providing CDN services for the domain being 
>>> fetched.
>>
>>
>>
>> I guess something wrong with NAT here when we use one port for intercept 
>> another for forwarding? I've done a lot of digging last 3 days, you may 
>> look at this( 
>> squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Transparent-intercept-Issues-td4681851.html 
>> ). I'm not sure this is relevant for my issue or not.
>>
>> *Thanks in advance.*
>>
>>  
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>>>
>>> It looks like you might need to add a firewall rule to allow 3127 to the 
>>> Grase server.
>>>
>>> However, without installing CA certificates on the client, what benefit 
>>> do you see from "proxying" HTTPS connections through squid?
>>>
>>> My long term plan was to remove Squid from future versions of the 
>>> hotspot, because the logs are becoming useless due to HTTPS traffic. I 
>>> believe the only thing you can get is the hostname it's connecting to, and 
>>> even that may not work with HTTPS 2.
>>>
>>> Regards
>>>
>>> Tim
>>>
>>> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com> wrote:
>>>
>>>> Hello,
>>>> I've managed to compile squid3 with SSL support in a standalone(squid3 
>>>> only) server, now I can monitor HTTPS traffic log without issuing any 
>>>> certificate. I got this idea originally from here: 
>>>> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>>>>
>>>> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
>>>> packages with SSL support, also kept all original Grase configurations. Now 
>>>> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
>>>> perfectly, squid restarted without any error.
>>>>
>>>> http_port 3128
>>>>> http_port 3129 intercept
>>>>> https_port 3127 intercept ssl-bump generate-host-certificates=off 
>>>>> cert=/etc/squid3/certs/squid.pem
>>>>> acl ssl-bump_port myportname 3127
>>>>> always_direct allow ssl-bump_port
>>>>
>>>>
>>>> For this new squid ports, default IPTABLE rules will not work. So I 
>>>> tried to modify* /etc/chilli/ipub.sh*
>>>>
>>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 
>>>>> -j DROP
>>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>>>> --dport 80 -j REDIRECT --to 3129
>>>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 
>>>>> -j DROP
>>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>>>> --dport 443 -j REDIRECT --to 3127
>>>>> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>>>>
>>>>
>>>> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* 
>>>> it somewhere!. If anyone can help to figure it out, that will be really a 
>>>> great-great help.
>>>>
>>>>
>>>> In my standalone server, I've used this IPTABLE rules and it works like 
>>>> a charm!
>>>>
>>>> *nat
>>>>> :PREROUTING ACCEPT
>>>>> :POSTROUTING ACCEPT
>>>>> :OUTPUT ACCEPT
>>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>>>> --dport 443 -j REDIRECT --to-ports 3127
>>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>>>> --dport 80 -j REDIRECT --to-ports 3129
>>>>> COMMIT
>>>>> *filter
>>>>> :INPUT DROP
>>>>> :FORWARD DROP
>>>>> :OUTPUT ACCEPT
>>>>> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>>>> -A INPUT -i lo -j ACCEPT
>>>>> -A OUTPUT -o lo -j ACCEPT
>>>>> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
>>>>> ACCEPT 
>>>>> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
>>>>> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
>>>>> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
>>>>> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
>>>>> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
>>>>> COMMIT
>>>>
>>>>
>>>>
>>>> *Thanks in advance.*
>>>>
>>>>
>>>>  
>>>>
>>>> -- 
>>>> This mailing list is for the Grase Hotspot Project 
>>>> http://grasehotspot.org
>>>> --- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Grase Hotspot" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to gr***.@grasehotspot.org.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
>>>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>

Thread