2019-09-10 - RE: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: Daniel Crusoe <di***n@gmail.com>
Message Hash: 894ce0cd6f5cf8630b0534747f8fbeb8dbc1549a3e08de25590bb1a6c30102f4
Message ID: <001601d5686b$7f307850$7d9168f0$@gmail.com>
Reply To: <2277ade4-9429-4821-8d07-d2f3f6dc387f@grasehotspot.org>
UTC Datetime: 2019-09-10 23:38:14 UTC
Raw Date: Wed, 11 Sep 2019 08:38:14 +0200

Raw message 

I am in no way knowledgeable in this, but, I have found that unless you give grase dns servers (not just on the outside NIC, but actually in the software) you have issues with connecting to any sites, and it is an intermittent issue.

 

Daniel 

 

From: SK NZ [mailto:sa***m@gmail.com] 
Sent: Wednesday, 11 September 2019 6:06 AM
To: Grase Hotspot <gr***t@grasehotspot.org>
Subject: Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

 

 

Hello Tim,

Thanks for the reply. I've edited the /etc/chilli/config to add port 3127, still no luck. I can't browse HTTP or HTTPS.

 

HS_TCP_PORTS="80 443 22 2812 53 3990 3127 3128" 

 

This is  iptables -S :

 

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -i eth1 -j DROP
-A INPUT -d 10.1.0.1/32 -i tun0 -p icmp -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 67:68 -j ACCEPT
-A INPUT -d 255.255.255.255/32 -i tun0 -p udp -m udp --dport 67:68 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3128 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3127 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 2812 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 4990 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
-A INPUT -d 10.1.0.1/32 -i tun0 -j DROP
-A FORWARD -i tun0 -o eth0 -j ACCEPT
-A FORWARD -i tun0 ! -o eth0 -j DROP
-A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -o tun0 -j ACCEPT
-A FORWARD -i tun0 -j ACCEPT
-A FORWARD -o eth1 -j DROP
-A FORWARD -i eth1 -j DROP

 

 

This is  iptables -vL : https://ibb.co/SVhTZHV

 

This is  /var/log/squid3/cache.log : https://paste.grasehotspot.org/view/cfa68b9a

 

I've tested squid3 locally in the terminal using squidclient, it works and logs for both - HTTP and HTTPS. So I guess it's not a squid issue.

 

192.168.0.103 TCP_MISS/200 909 POST http://ocsp.digicert.com/ - HIER_DIRECT/117.18.237.29 application/ocsp-response
192.168.0.103 TCP_MISS/200 29835 CONNECT github.githubassets.com:443 - HIER_DIRECT/185.199.109.154 -
192.168.0.103 TCP_MISS/200 81152 CONNECT github.githubassets.com:443 - HIER_DIRECT/185.199.109.154 -
192.168.0.103 TCP_MISS/200 22041 CONNECT github.githubassets.com:443 - HIER_DIRECT/185.199.109.154 -
192.168.0.103 TCP_MISS/200 37913 CONNECT customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
192.168.0.103 TCP_MISS/200 571167 CONNECT customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
192.168.0.103 TCP_MISS/200 3741 CONNECT www.google-analytics.com:443 <http://www.google-analytics.com:443>  - HIER_DIRECT/74.125.68.139 -

 


Yes, you're right, for HTTPS without issuing certificate we'll only get hostnames. Above log is for https-github. At least now we can tell which user is connecting to which https sites, it's better than nothing. I can't provide free public wifi without keeping logs, it's our local compliance. So I've to keep at least some form of logs. Please help me to figure it out the issue. Thanks in advance.

 






On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:

It looks like you might need to add a firewall rule to allow 3127 to the Grase server.

 

However, without installing CA certificates on the client, what benefit do you see from "proxying" HTTPS connections through squid?

 

My long term plan was to remove Squid from future versions of the hotspot, because the logs are becoming useless due to HTTPS traffic. I believe the only thing you can get is the hostname it's connecting to, and even that may not work with HTTPS 2.

 

Regards

 

Tim

 

On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com <javascript:> > wrote:

Hello,
I've managed to compile squid3 with SSL support in a standalone(squid3 only) server, now I can monitor HTTPS traffic log without issuing any certificate. I got this idea originally from here: http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html

To implement this in a Grase Hotspot Server, I reinstalled squid3 packages with SSL support, also kept all original Grase configurations. Now I modified squid.conf.grase to enable HTTPS. So far it worked perfectly, squid restarted without any error.

 

http_port 3128
http_port 3129 intercept
https_port 3127 intercept ssl-bump generate-host-certificates=off cert=/etc/squid3/certs/squid.pem
acl ssl-bump_port myportname 3127
always_direct allow ssl-bump_port


For this new squid ports, default IPTABLE rules will not work. So I tried to modify /etc/chilli/ipub.sh

ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 -j DROP
ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR --dport 80 -j REDIRECT --to 3129
ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j DROP
ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR --dport 443 -j REDIRECT --to 3127
ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE


Now I can't browse after connecting hotspot. IPTABLE is blocking it somewhere!. If anyone can help to figure it out, that will be really a great-great help.


In my standalone server, I've used this IPTABLE rules and it works like a charm!

*nat
:PREROUTING ACCEPT
:POSTROUTING ACCEPT
:OUTPUT ACCEPT
-A PREROUTING -i eth0 -s 192.168.0.0/16 <http://192.168.0.0/16>  ! -d 192.168.0.0/16 <http://192.168.0.0/16>  -p tcp --dport 443 -j REDIRECT --to-ports 3127
-A PREROUTING -i eth0 -s 192.168.0.0/16 <http://192.168.0.0/16>  ! -d 192.168.0.0/16 <http://192.168.0.0/16>  -p tcp --dport 80 -j REDIRECT --to-ports 3129
COMMIT
*filter
:INPUT DROP
:FORWARD DROP
:OUTPUT ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT 
-A INPUT -i eth0 -p tcp --dport http -j ACCEPT
-A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
-A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
-A INPUT -i eth0 -p udp --dport domain -j ACCEPT
-A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
COMMIT



Thanks in advance.


 

-- 
This mailing list is for the Grase Hotspot Project http://grasehotspot.org
--- 
You received this message because you are subscribed to the Google Groups "Grase Hotspot" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gr***.@grasehotspot.org <javascript:> .
To view this discussion on the web visit https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer> .

-- 
This mailing list is for the Grase Hotspot Project http://grasehotspot.org
--- 
You received this message because you are subscribed to the Google Groups "Grase Hotspot" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gr***e@grasehotspot.org <mailto:gr***e@grasehotspot.org> .
To view this discussion on the web visit https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org?utm_medium=email&utm_source=footer> .

Thread