2019-09-10 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: SK NZ <sa***m@gmail.com>
Message Hash: a5b25bd7f9a14ef2a1adaa223e9171d57baef0dea65fb96c3be09f9b135bc7d5
Message ID: <2277ade4-9429-4821-8d07-d2f3f6dc387f@grasehotspot.org>
Reply To: <CAESLx0LcePJj_V-kGgrkiYOQdpZndZG9zYb2-mFZ-COp8hXSjA@mail.gmail.com>
UTC Datetime: 2019-09-10 21:05:36 UTC
Raw Date: Tue, 10 Sep 2019 21:05:36 -0700

Raw message 



Hello Tim,
Thanks for the reply. I've edited the */etc/chilli/config* to add port 
*3127*, still no luck. I can't browse HTTP or HTTPS.

HS_TCP_PORTS="80 443 22 2812 53 3990 *3127* 3128" 



This is  *iptables -S* :

-P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -A INPUT -i eth1 -j DROP
> -A INPUT -d 10.1.0.1/32 -i tun0 -p icmp -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 67:68 -j ACCEPT
> -A INPUT -d 255.255.255.255/32 -i tun0 -p udp -m udp --dport 67:68 -j 
> ACCEPT
> *-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3128 -j ACCEPT*
> *-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3127 -j ACCEPT*
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 2812 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 22 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 443 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 80 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 4990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -j DROP
> -A FORWARD -i tun0 -o eth0 -j ACCEPT
> -A FORWARD -i tun0 ! -o eth0 -j DROP
> -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS 
> --clamp-mss-to-pmtu
> -A FORWARD -o tun0 -j ACCEPT
> -A FORWARD -i tun0 -j ACCEPT
> -A FORWARD -o eth1 -j DROP
> -A FORWARD -i eth1 -j DROP



This is  *iptables -vL* : https://ibb.co/SVhTZHV

This is  */var/log/squid3/cache.log* : 
https://paste.grasehotspot.org/view/cfa68b9a

I've tested squid3 locally in the terminal using squidclient, it works and 
logs for both - HTTP and HTTPS. So I guess it's not a squid issue.


192.168.0.103 TCP_MISS/200 909 POST http://ocsp.digicert.com/ - 
> HIER_DIRECT/117.18.237.29 application/ocsp-response
> 192.168.0.103 TCP_MISS/200 29835 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 81152 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 22041 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 37913 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 571167 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 3741 CONNECT www.google-analytics.com:443 - 
> HIER_DIRECT/74.125.68.139 -



Yes, you're right, for HTTPS without issuing certificate we'll only get 
hostnames. Above log is for https-github. At least now we can tell which 
user is connecting to which https sites, it's better than nothing. I can't 
provide free public wifi without keeping logs, it's our local compliance. 
So I've to keep at least some form of logs. Please help me to figure it out 
the issue. Thanks in advance.






On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>
> It looks like you might need to add a firewall rule to allow 3127 to the 
> Grase server.
>
> However, without installing CA certificates on the client, what benefit do 
> you see from "proxying" HTTPS connections through squid?
>
> My long term plan was to remove Squid from future versions of the hotspot, 
> because the logs are becoming useless due to HTTPS traffic. I believe the 
> only thing you can get is the hostname it's connecting to, and even that 
> may not work with HTTPS 2.
>
> Regards
>
> Tim
>
> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com <javascript:>> 
> wrote:
>
>> Hello,
>> I've managed to compile squid3 with SSL support in a standalone(squid3 
>> only) server, now I can monitor HTTPS traffic log without issuing any 
>> certificate. I got this idea originally from here: 
>> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>>
>> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
>> packages with SSL support, also kept all original Grase configurations. Now 
>> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
>> perfectly, squid restarted without any error.
>>
>> http_port 3128
>>> http_port 3129 intercept
>>> https_port 3127 intercept ssl-bump generate-host-certificates=off 
>>> cert=/etc/squid3/certs/squid.pem
>>> acl ssl-bump_port myportname 3127
>>> always_direct allow ssl-bump_port
>>
>>
>> For this new squid ports, default IPTABLE rules will not work. So I tried 
>> to modify* /etc/chilli/ipub.sh*
>>
>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 -j 
>>> DROP
>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>> --dport 80 -j REDIRECT --to 3129
>>> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 
>>> -j DROP
>>> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
>>> --dport 443 -j REDIRECT --to 3127
>>> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>>
>>
>> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it 
>> somewhere!. If anyone can help to figure it out, that will be really a 
>> great-great help.
>>
>>
>> In my standalone server, I've used this IPTABLE rules and it works like a 
>> charm!
>>
>> *nat
>>> :PREROUTING ACCEPT
>>> :POSTROUTING ACCEPT
>>> :OUTPUT ACCEPT
>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>> --dport 443 -j REDIRECT --to-ports 3127
>>> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
>>> --dport 80 -j REDIRECT --to-ports 3129
>>> COMMIT
>>> *filter
>>> :INPUT DROP
>>> :FORWARD DROP
>>> :OUTPUT ACCEPT
>>> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>> -A INPUT -i lo -j ACCEPT
>>> -A OUTPUT -o lo -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
>>> ACCEPT 
>>> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
>>> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
>>> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
>>> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
>>> COMMIT
>>
>>
>>
>> *Thanks in advance.*
>>
>>
>>  
>>
>> -- 
>> This mailing list is for the Grase Hotspot Project 
>> http://grasehotspot.org
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Grase Hotspot" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to gr***.@grasehotspot.org <javascript:>.
>> To view this discussion on the web visit 
>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
>> .
>>
>

Thread