2019-09-17 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: Tim <ti***8@gmail.com>
Message Hash: 644efd9342465b52e3792e230d032ca2e02c599d573c0600925e30d1d296271e
Message ID: <CAESLx0+1Vtg2nTRBw98uz05rcOFPnEbxsynuMJVN0Xi+kWht6A@mail.gmail.com>
Reply To: <08020c3b-e607-46fc-9f3b-0cc50e26bb41@grasehotspot.org>
UTC Datetime: 2019-09-17 13:54:20 UTC
Raw Date: Wed, 18 Sep 2019 06:54:20 +1000

Raw message 

Hi SK

At this stage Debian Jessie isn't a supported distribution. Obviously
things have changed with the Squid reverse proxy. Work is currently
progressing on V4 of the hotspot, I suggest until then you try installing
on Debian 7, or Ubuntu 14.04. Alternatively, you can remove Squid from the
config and have everything working without the Squid proxy. Due to the high
number of SSL sites in 2019, Squid caching is not going to be very
beneficial to you anymore.

Regards

Tim

On Thu, 12 Sep 2019 at 23:49, SK NZ <sa***m@gmail.com> wrote:

> Hello Tim,
> I'm stuck with this issue for the last 3 days. If you kindly give a look
> at it that will be really life-saving! I've added port *3127* in the
> firewall rules as you suggested but no luck yet! *I cannot browse HTTP or
> HTTPS sites under AP hotspot*. It's a fresh grase hotspot server, minor
> modifications are listed below.
>
> *iptables, squid, chili rules:*
> https://paste.grasehotspot.org/view/raw/80fcc90f
>
> After a fresh boot, I can test squid from ssh(*squidclient
> https://grasehotspot.org <https://grasehotspot.org>*). It works until 1st
> user login using AP hotspot, after that squidclient also stops responding.
> I've checked squid3 status, it says *active(running)*. So I guess
> iptables or chilli is doing something to squid, that's weird!
>
> Exact same squid configuration works in the same version of squid and OS
> in a standalone server, but it's not working here. I believe the only
> difference is iptable/nat rules. *Another worthy thing to mention,* after
> a fresh grase hotspot server installation it doesn't work out of the box.
> For default squid.conf.grase, it throws *Error: No forward-proxy ports
> configure. *So I have to add a forward port.
>
> http_port 3128
>> http_port 3127 intercept
>
>
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j
>> DROP
>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR
>> --dport 80 -j REDIRECT --to 3127
>
>
> I'm using stable *Squid 3.4.8 @ Debian Jessie*, I guess from this version
> it's a must to specify a forward port. This stops throwing an error in
> squid cache log, still, HTTP sites are not working. It shows *ERROR/Invalid
> URL* squid error page. So I have to modify *squid.conf.grase* again to
> make it work!
>
> http_port 3128 accel vhost allow-direct
>
>
> It's not a transparent proxy anymore, rather a reverse proxy. Now HTTP
> sites are working and also monitor session works in *grase/radmin*. *So
> INTERCEPT is not working for only HTTP or both.*
>
> intercept means take the origin server details from the NAT system.
>> vs.
>> accel means Squid is providing CDN services for the domain being fetched.
>
>
>
> I guess something wrong with NAT here when we use one port for intercept
> another for forwarding? I've done a lot of digging last 3 days, you may
> look at this(
> squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Transparent-intercept-Issues-td4681851.html
> ). I'm not sure this is relevant for my issue or not.
>
> *Thanks in advance.*
>
>
>
>
>
>
>
>
>
>
>
> On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>>
>> It looks like you might need to add a firewall rule to allow 3127 to the
>> Grase server.
>>
>> However, without installing CA certificates on the client, what benefit
>> do you see from "proxying" HTTPS connections through squid?
>>
>> My long term plan was to remove Squid from future versions of the
>> hotspot, because the logs are becoming useless due to HTTPS traffic. I
>> believe the only thing you can get is the hostname it's connecting to, and
>> even that may not work with HTTPS 2.
>>
>> Regards
>>
>> Tim
>>
>> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com> wrote:
>>
>>> Hello,
>>> I've managed to compile squid3 with SSL support in a standalone(squid3
>>> only) server, now I can monitor HTTPS traffic log without issuing any
>>> certificate. I got this idea originally from here:
>>> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>>>
>>> *To implement this in a Grase Hotspot Server*, I reinstalled squid3
>>> packages with SSL support, also kept all original Grase configurations. Now
>>> I modified *squid.conf.grase* to enable HTTPS. So far it worked
>>> perfectly, squid restarted without any error.
>>>
>>> http_port 3128
>>>> http_port 3129 intercept
>>>> https_port 3127 intercept ssl-bump generate-host-certificates=off
>>>> cert=/etc/squid3/certs/squid.pem
>>>> acl ssl-bump_port myportname 3127
>>>> always_direct allow ssl-bump_port
>>>
>>>
>>> For this new squid ports, default IPTABLE rules will not work. So I
>>> tried to modify* /etc/chilli/ipub.sh*
>>>
>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129
>>>> -j DROP
>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR
>>>> --dport 80 -j REDIRECT --to 3129
>>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127
>>>> -j DROP
>>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR
>>>> --dport 443 -j REDIRECT --to 3127
>>>> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>>>
>>>
>>> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it
>>> somewhere!. If anyone can help to figure it out, that will be really a
>>> great-great help.
>>>
>>>
>>> In my standalone server, I've used this IPTABLE rules and it works like
>>> a charm!
>>>
>>> *nat
>>>> :PREROUTING ACCEPT
>>>> :POSTROUTING ACCEPT
>>>> :OUTPUT ACCEPT
>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp
>>>> --dport 443 -j REDIRECT --to-ports 3127
>>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp
>>>> --dport 80 -j REDIRECT --to-ports 3129
>>>> COMMIT
>>>> *filter
>>>> :INPUT DROP
>>>> :FORWARD DROP
>>>> :OUTPUT ACCEPT
>>>> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>>> -A INPUT -i lo -j ACCEPT
>>>> -A OUTPUT -o lo -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j
>>>> ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
>>>> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
>>>> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
>>>> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
>>>> COMMIT
>>>
>>>
>>>
>>> *Thanks in advance.*
>>>
>>>
>>>
>>>
>>> --
>>> This mailing list is for the Grase Hotspot Project
>>> http://grasehotspot.org
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Grase Hotspot" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to gr***.@grasehotspot.org.
>>> To view this discussion on the web visit
>>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org
>>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
> This mailing list is for the Grase Hotspot Project http://grasehotspot.org
> ---
> You received this message because you are subscribed to the Google Groups
> "Grase Hotspot" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gr***e@grasehotspot.org.
> To view this discussion on the web visit
> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/08020c3b-e607-46fc-9f3b-0cc50e26bb41%40grasehotspot.org
> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/08020c3b-e607-46fc-9f3b-0cc50e26bb41%40grasehotspot.org?utm_medium=email&utm_source=footer>
> .
>

Thread