2019-09-12 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: christopher <me***e@pc-networking-services.com>
Message Hash: 71435b08fa71d10b0f5d7ba000285fd0096373dc7f979a945ab3af2f66252fdc
Message ID: <b5f30b5c-0e52-42a2-bd57-d1f2df78ab3c@grasehotspot.org>
Reply To: <08020c3b-e607-46fc-9f3b-0cc50e26bb41@grasehotspot.org>
UTC Datetime: 2019-09-12 09:30:31 UTC
Raw Date: Thu, 12 Sep 2019 09:30:31 -0700

Raw message 

Hello,

I just did a quick look on the net, and the following may give you the 
solution:

https://serverfault.com/questions/870319/ip-tables-intercept-exception

Regards,

Christopher.

On Friday, 13 September 2019 01:49:48 UTC+12, SK NZ wrote:
>
> Hello Tim,
> I'm stuck with this issue for the last 3 days. If you kindly give a look 
> at it that will be really life-saving! I've added port *3127* in the 
> firewall rules as you suggested but no luck yet! *I cannot browse HTTP or 
> HTTPS sites under AP hotspot*. It's a fresh grase hotspot server, minor 
> modifications are listed below.
>
> *iptables, squid, chili rules:* 
> https://paste.grasehotspot.org/view/raw/80fcc90f
>
> After a fresh boot, I can test squid from ssh(*squidclient  
> https://grasehotspot.org <https://grasehotspot.org>*). It works until 1st 
> user login using AP hotspot, after that squidclient also stops responding. 
> I've checked squid3 status, it says *active(running)*. So I guess 
> iptables or chilli is doing something to squid, that's weird!
>
> Exact same squid configuration works in the same version of squid and OS 
> in a standalone server, but it's not working here. I believe the only 
> difference is iptable/nat rules. *Another worthy thing to mention,* after 
> a fresh grase hotspot server installation it doesn't work out of the box. 
> For default squid.conf.grase, it throws *Error: No forward-proxy ports 
> configure. *So I have to add a forward port.
>
> http_port 3128
>> http_port 3127 intercept
>
>
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
>> DROP
>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>> --dport 80 -j REDIRECT --to 3127
>
>
> I'm using stable *Squid 3.4.8 @ Debian Jessie*, I guess from this version 
> it's a must to specify a forward port. This stops throwing an error in 
> squid cache log, still, HTTP sites are not working. It shows *ERROR/Invalid 
> URL* squid error page. So I have to modify *squid.conf.grase* again to 
> make it work!
>
> http_port 3128 accel vhost allow-direct
>
>
> It's not a transparent proxy anymore, rather a reverse proxy. Now HTTP 
> sites are working and also monitor session works in *grase/radmin*. *So 
> INTERCEPT is not working for only HTTP or both.*
>
> intercept means take the origin server details from the NAT system.
>> vs.
>> accel means Squid is providing CDN services for the domain being fetched.
>
>
>
> I guess something wrong with NAT here when we use one port for intercept 
> another for forwarding? I've done a lot of digging last 3 days, you may 
> look at this( 
> squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Transparent-intercept-Issues-td4681851.html 
> ). I'm not sure this is relevant for my issue or not.
>
> *Thanks in advance.*
>
>  
>
>
>
>
>
>
>
>
>
> On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>>
>> It looks like you might need to add a firewall rule to allow 3127 to the 
>> Grase server.
>>
>> However, without installing CA certificates on the client, what benefit 
>> do you see from "proxying" HTTPS connections through squid?
>>
>> My long term plan was to remove Squid from future versions of the 
>> hotspot, because the logs are becoming useless due to HTTPS traffic. I 
>> believe the only thing you can get is the hostname it's connecting to, and 
>> even that may not work with HTTPS 2.
>>
>> Regards
>>
>> Tim
>>
>> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com> wrote:
>>
>>> Hello,
>>> I've managed to compile squid3 with SSL support in a standalone(squid3 
>>> only) server, now I can monitor HTTPS traffic log without issuing any 
>>> certificate. I got this idea originally from here: 
>>> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>>>
>>> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
>>> packages with SSL support, also kept all original Grase configurations. Now 
>>> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
>>> perfectly, squid restarted without any error.
>>>
>>> http_port 3128
>>>> http_port 3129 intercept
>>>> https_port 3127 intercept ssl-bump generate-host-certificates=off 
>>>> cert=/etc/squid3/certs/squid.pem
>>>> acl ssl-bump_port myportname 3127
>>>> always_direct allow ssl-bump_port
>>>
>>>
>>> For this new squid ports, default IPTABLE rules will not work. So I 
>>> tried to modify* /etc/chilli/ipub.sh*
>>>
>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 
>>>> -j DROP
>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>>> --dport 80 -j REDIRECT --to 3129
>>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 
>>>> -j DROP
>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>>> --dport 443 -j REDIRECT --to 3127
>>>> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>>>
>>>
>>> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it 
>>> somewhere!. If anyone can help to figure it out, that will be really a 
>>> great-great help.
>>>
>>>
>>> In my standalone server, I've used this IPTABLE rules and it works like 
>>> a charm!
>>>
>>> *nat
>>>> :PREROUTING ACCEPT
>>>> :POSTROUTING ACCEPT
>>>> :OUTPUT ACCEPT
>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>>> --dport 443 -j REDIRECT --to-ports 3127
>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>>> --dport 80 -j REDIRECT --to-ports 3129
>>>> COMMIT
>>>> *filter
>>>> :INPUT DROP
>>>> :FORWARD DROP
>>>> :OUTPUT ACCEPT
>>>> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>>> -A INPUT -i lo -j ACCEPT
>>>> -A OUTPUT -o lo -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
>>>> ACCEPT 
>>>> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
>>>> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
>>>> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
>>>> COMMIT
>>>
>>>
>>>
>>> *Thanks in advance.*
>>>
>>>
>>>  
>>>
>>> -- 
>>> This mailing list is for the Grase Hotspot Project 
>>> http://grasehotspot.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Grase Hotspot" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to gr***.@grasehotspot.org.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
>>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
>>> .
>>>
>>

Thread