2019-09-10 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: SK NZ <sa***m@gmail.com>
Message Hash: a9a0146ce9b47ae469a0ae78d161704b61cfea9c226902ca9d103d8373dc103d
Message ID: <5e30e94b-b4c3-4351-9423-5b61c387b428@grasehotspot.org>
Reply To: <001601d5686b$7f307850$7d9168f0$@gmail.com>
UTC Datetime: 2019-09-10 23:44:03 UTC
Raw Date: Tue, 10 Sep 2019 23:44:03 -0700

Raw message 

Hello Daniel,
Can you please elaborate what do you mean by *give grase dns servers*? or 
how to achieve this? Thanks.



On Wednesday, September 11, 2019 at 12:38:25 PM UTC+6, Daniel Crusoe wrote:
>
> I am in no way knowledgeable in this, but, I have found that unless you 
> give grase dns servers (not just on the outside NIC, but actually in the 
> software) you have issues with connecting to any sites, and it is an 
> intermittent issue.
>
>  
>
> Daniel 
>
>  
>
> *From:* SK NZ [mailto:sa***.@gmail.com <javascript:>] 
> *Sent:* Wednesday, 11 September 2019 6:06 AM
> *To:* Grase Hotspot <gr***.@grasehotspot.org <javascript:>>
> *Subject:* Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue
>
>  
>
>  
>
> Hello Tim,
>
> Thanks for the reply. I've edited the */etc/chilli/config* to add port 
> *3127*, still no luck. I can't browse HTTP or HTTPS.
>
>  
>
> HS_TCP_PORTS="80 443 22 2812 53 3990 *3127* 3128" 
>
>  
>
> This is  *iptables -S* :
>
>  
>
> -P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -A INPUT -i eth1 -j DROP
> -A INPUT -d 10.1.0.1/32 -i tun0 -p icmp -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 67:68 -j ACCEPT
> -A INPUT -d 255.255.255.255/32 -i tun0 -p udp -m udp --dport 67:68 -j 
> ACCEPT
> *-A INPUT -d 10.1.0.1/32 <http://10.1.0.1/32> -i tun0 -p tcp -m tcp 
> --dport 3128 -j ACCEPT*
> *-A INPUT -d 10.1.0.1/32 <http://10.1.0.1/32> -i tun0 -p tcp -m tcp 
> --dport 3127 -j ACCEPT*
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 2812 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 22 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 443 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 80 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 4990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -j DROP
> -A FORWARD -i tun0 -o eth0 -j ACCEPT
> -A FORWARD -i tun0 ! -o eth0 -j DROP
> -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS 
> --clamp-mss-to-pmtu
> -A FORWARD -o tun0 -j ACCEPT
> -A FORWARD -i tun0 -j ACCEPT
> -A FORWARD -o eth1 -j DROP
> -A FORWARD -i eth1 -j DROP
>
>  
>
>  
>
> This is  *iptables -vL* : https://ibb.co/SVhTZHV
>
>  
>
> This is  */var/log/squid3/cache.log* : 
> https://paste.grasehotspot.org/view/cfa68b9a
>
>  
>
> I've tested squid3 locally in the terminal using squidclient, it works and 
> logs for both - HTTP and HTTPS. So I guess it's not a squid issue.
>
>  
>
> 192.168.0.103 TCP_MISS/200 909 POST http://ocsp.digicert.com/ - 
> HIER_DIRECT/117.18.237.29 application/ocsp-response
> 192.168.0.103 TCP_MISS/200 29835 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 81152 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 22041 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 37913 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 571167 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 3741 CONNECT www.google-analytics.com:443 - 
> HIER_DIRECT/74.125.68.139 -
>
>  
>
>
> Yes, you're right, for HTTPS without issuing certificate we'll only get 
> hostnames. Above log is for https-github. At least now we can tell which 
> user is connecting to which https sites, it's better than nothing. I can't 
> provide free public wifi without keeping logs, it's our local compliance. 
> So I've to keep at least some form of logs. Please help me to figure it out 
> the issue. Thanks in advance.
>
>  
>
>
>
>
>
>
> On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>
> It looks like you might need to add a firewall rule to allow 3127 to the 
> Grase server.
>
>  
>
> However, without installing CA certificates on the client, what benefit do 
> you see from "proxying" HTTPS connections through squid?
>
>  
>
> My long term plan was to remove Squid from future versions of the hotspot, 
> because the logs are becoming useless due to HTTPS traffic. I believe the 
> only thing you can get is the hostname it's connecting to, and even that 
> may not work with HTTPS 2.
>
>  
>
> Regards
>
>  
>
> Tim
>
>  
>
> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com> wrote:
>
> Hello,
> I've managed to compile squid3 with SSL support in a standalone(squid3 
> only) server, now I can monitor HTTPS traffic log without issuing any 
> certificate. I got this idea originally from here: 
> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>
> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
> packages with SSL support, also kept all original Grase configurations. Now 
> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
> perfectly, squid restarted without any error.
>
>  
>
> http_port 3128
> http_port 3129 intercept
> https_port 3127 intercept ssl-bump generate-host-certificates=off 
> cert=/etc/squid3/certs/squid.pem
> acl ssl-bump_port myportname 3127
> always_direct allow ssl-bump_port
>
>
> For this new squid ports, default IPTABLE rules will not work. So I tried 
> to modify* /etc/chilli/ipub.sh*
>
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 80 -j REDIRECT --to 3129
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 443 -j REDIRECT --to 3127
> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>
>
> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it 
> somewhere!. If anyone can help to figure it out, that will be really a 
> great-great help.
>
>
> In my standalone server, I've used this IPTABLE rules and it works like a 
> charm!
>
> *nat
> :PREROUTING ACCEPT
> :POSTROUTING ACCEPT
> :OUTPUT ACCEPT
> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
> --dport 443 -j REDIRECT --to-ports 3127
> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
> --dport 80 -j REDIRECT --to-ports 3129
> COMMIT
> *filter
> :INPUT DROP
> :FORWARD DROP
> :OUTPUT ACCEPT
> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A OUTPUT -o lo -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
> ACCEPT 
> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
> COMMIT
>
>
>
> *Thanks in advance.*
>
>
>  
>
> -- 
> This mailing list is for the Grase Hotspot Project http://grasehotspot.org
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Grase Hotspot" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to gr***.@grasehotspot.org.
> To view this discussion on the web visit 
> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
> .
>
> -- 
> This mailing list is for the Grase Hotspot Project http://grasehotspot.org
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Grase Hotspot" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to gr***.@grasehotspot.org <javascript:>.
> To view this discussion on the web visit 
> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org 
> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org?utm_medium=email&utm_source=footer>
> .
>

Thread