2019-09-11 - Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue

Header Data

From: christopher <me***e@pc-networking-services.com>
Message Hash: 96e4a71be78e150a54ccce0e4a172e34fba20509167db04154c970efbcc5f6ec
Message ID: <dd66f64e-29e6-4c84-9063-cd887aa6a4e7@grasehotspot.org>
Reply To: <001601d5686b$7f307850$7d9168f0$@gmail.com>
UTC Datetime: 2019-09-11 04:37:16 UTC
Raw Date: Wed, 11 Sep 2019 04:37:16 -0700

Raw message 

Hello Daniel,

I have no idea where you got this idea from.  

I have had grase hotspot installed since 2016 and I have ONLY added the ip 
address of the router connected to the WAN side and google's public 
nameserver address in the admin panel of grase, and it has NEVER failed.  

I have been a system administrator for decades, and have used a large 
number of the linux distros out there at one time or another.  If you are 
running a mail server you need a dns server.  For a hotspot server that is 
not publicly serving any internet accessible website, no other 
configuration is needed, other than to have what is needed for grase.

Christopher.

On Wednesday, 11 September 2019 18:38:25 UTC+12, Daniel Crusoe wrote:
>
> I am in no way knowledgeable in this, but, I have found that unless you 
> give grase dns servers (not just on the outside NIC, but actually in the 
> software) you have issues with connecting to any sites, and it is an 
> intermittent issue.
>
>  
>
> Daniel 
>
>  
>
> *From:* SK NZ [mailto:sa***.@gmail.com <javascript:>] 
> *Sent:* Wednesday, 11 September 2019 6:06 AM
> *To:* Grase Hotspot <gr***.@grasehotspot.org <javascript:>>
> *Subject:* Re: [GRASE-Hotspot] HTTPS Traffic Log IPTABLE Issue
>
>  
>
>  
>
> Hello Tim,
>
> Thanks for the reply. I've edited the */etc/chilli/config* to add port 
> *3127*, still no luck. I can't browse HTTP or HTTPS.
>
>  
>
> HS_TCP_PORTS="80 443 22 2812 53 3990 *3127* 3128" 
>
>  
>
> This is  *iptables -S* :
>
>  
>
> -P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -A INPUT -i eth1 -j DROP
> -A INPUT -d 10.1.0.1/32 -i tun0 -p icmp -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p udp -m udp --dport 67:68 -j ACCEPT
> -A INPUT -d 255.255.255.255/32 -i tun0 -p udp -m udp --dport 67:68 -j 
> ACCEPT
> *-A INPUT -d 10.1.0.1/32 <http://10.1.0.1/32> -i tun0 -p tcp -m tcp 
> --dport 3128 -j ACCEPT*
> *-A INPUT -d 10.1.0.1/32 <http://10.1.0.1/32> -i tun0 -p tcp -m tcp 
> --dport 3127 -j ACCEPT*
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 2812 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 22 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 443 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 80 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 4990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -p tcp -m tcp --dport 3990 -j ACCEPT
> -A INPUT -d 10.1.0.1/32 -i tun0 -j DROP
> -A FORWARD -i tun0 -o eth0 -j ACCEPT
> -A FORWARD -i tun0 ! -o eth0 -j DROP
> -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS 
> --clamp-mss-to-pmtu
> -A FORWARD -o tun0 -j ACCEPT
> -A FORWARD -i tun0 -j ACCEPT
> -A FORWARD -o eth1 -j DROP
> -A FORWARD -i eth1 -j DROP
>
>  
>
>  
>
> This is  *iptables -vL* : https://ibb.co/SVhTZHV
>
>  
>
> This is  */var/log/squid3/cache.log* : 
> https://paste.grasehotspot.org/view/cfa68b9a
>
>  
>
> I've tested squid3 locally in the terminal using squidclient, it works and 
> logs for both - HTTP and HTTPS. So I guess it's not a squid issue.
>
>  
>
> 192.168.0.103 TCP_MISS/200 909 POST http://ocsp.digicert.com/ - 
> HIER_DIRECT/117.18.237.29 application/ocsp-response
> 192.168.0.103 TCP_MISS/200 29835 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 81152 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 22041 CONNECT github.githubassets.com:443 - 
> HIER_DIRECT/185.199.109.154 -
> 192.168.0.103 TCP_MISS/200 37913 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 571167 CONNECT 
> customer-stories-feed.github.com:443 - HIER_DIRECT/185.199.110.153 -
> 192.168.0.103 TCP_MISS/200 3741 CONNECT www.google-analytics.com:443 - 
> HIER_DIRECT/74.125.68.139 -
>
>  
>
>
> Yes, you're right, for HTTPS without issuing certificate we'll only get 
> hostnames. Above log is for https-github. At least now we can tell which 
> user is connecting to which https sites, it's better than nothing. I can't 
> provide free public wifi without keeping logs, it's our local compliance. 
> So I've to keep at least some form of logs. Please help me to figure it out 
> the issue. Thanks in advance.
>
>  
>
>
>
>
>
>
> On Wednesday, September 11, 2019 at 3:06:23 AM UTC+6, timwhite88 wrote:
>
> It looks like you might need to add a firewall rule to allow 3127 to the 
> Grase server.
>
>  
>
> However, without installing CA certificates on the client, what benefit do 
> you see from "proxying" HTTPS connections through squid?
>
>  
>
> My long term plan was to remove Squid from future versions of the hotspot, 
> because the logs are becoming useless due to HTTPS traffic. I believe the 
> only thing you can get is the hostname it's connecting to, and even that 
> may not work with HTTPS 2.
>
>  
>
> Regards
>
>  
>
> Tim
>
>  
>
> On Tue, 10 Sep 2019 at 21:24, SK NZ <sa***.@gmail.com> wrote:
>
> Hello,
> I've managed to compile squid3 with SSL support in a standalone(squid3 
> only) server, now I can monitor HTTPS traffic log without issuing any 
> certificate. I got this idea originally from here: 
> http://blog.manty.net/2014/12/squid-proxy-being-transparent-also-for.html
>
> *To implement this in a Grase Hotspot Server*, I reinstalled squid3 
> packages with SSL support, also kept all original Grase configurations. Now 
> I modified *squid.conf.grase* to enable HTTPS. So far it worked 
> perfectly, squid restarted without any error.
>
>  
>
> http_port 3128
> http_port 3129 intercept
> https_port 3127 intercept ssl-bump generate-host-certificates=off 
> cert=/etc/squid3/certs/squid.pem
> acl ssl-bump_port myportname 3127
> always_direct allow ssl-bump_port
>
>
> For this new squid ports, default IPTABLE rules will not work. So I tried 
> to modify* /etc/chilli/ipub.sh*
>
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3129 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 80 -j REDIRECT --to 3129
> ipt -I PREROUTING -t mangle -p tcp -s $NET/$MASK -d $ADDR --dport 3127 -j 
> DROP
> ipt -I PREROUTING -t nat -i $TUNTAP -p tcp -s $NET/$MASK ! -d $ADDR 
> --dport 443 -j REDIRECT --to 3127
> ipt -I POSTROUTING -t nat -o $HS_WANIF -j MASQUERADE
>
>
> Now I can't browse after connecting hotspot. *IPTABLE* *is blocking* it 
> somewhere!. If anyone can help to figure it out, that will be really a 
> great-great help.
>
>
> In my standalone server, I've used this IPTABLE rules and it works like a 
> charm!
>
> *nat
> :PREROUTING ACCEPT
> :POSTROUTING ACCEPT
> :OUTPUT ACCEPT
> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
> --dport 443 -j REDIRECT --to-ports 3127
> -A PREROUTING -i eth0 -s 192.168.0.0/16 ! -d 192.168.0.0/16 -p tcp 
> --dport 80 -j REDIRECT --to-ports 3129
> COMMIT
> *filter
> :INPUT DROP
> :FORWARD DROP
> :OUTPUT ACCEPT
> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A OUTPUT -o lo -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j 
> ACCEPT 
> -A INPUT -i eth0 -p tcp --dport http -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport 3127:3128 -j ACCEPT
> -A INPUT -i eth0 -p udp --dport bootps -j ACCEPT
> -A INPUT -i eth0 -p udp --dport domain -j ACCEPT
> -A INPUT -i eth0 -p tcp --dport domain -j ACCEPT
> COMMIT
>
>
>
> *Thanks in advance.*
>
>
>  
>
> -- 
> This mailing list is for the Grase Hotspot Project http://grasehotspot.org
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Grase Hotspot" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to gr***.@grasehotspot.org.
> To view this discussion on the web visit 
> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org 
> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/f6deda79-b57c-4243-aa26-be9952b2eba6%40grasehotspot.org?utm_medium=email&utm_source=footer>
> .
>
> -- 
> This mailing list is for the Grase Hotspot Project http://grasehotspot.org
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Grase Hotspot" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to gr***.@grasehotspot.org <javascript:>.
> To view this discussion on the web visit 
> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org 
> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/2277ade4-9429-4821-8d07-d2f3f6dc387f%40grasehotspot.org?utm_medium=email&utm_source=footer>
> .
>

Thread