2017-04-17 - Re: [GRASE-Hotspot] Semi-Urgent - coova-chilli[1382]: net.c: 780: 11 (Resource temporarily unavailable) could not read packet

Header Data

From: José Borges <jo***s@algardata.pt>
Message Hash: c6301b3cf81c5c3be5c7e6a4ab818f754bb10f2e384a1437e0e7f294c342710a
Message ID: <f6271313-2e9a-49dc-8887-8a14983ae95d@grasehotspot.org>
Reply To: <dbafe41d-3766-4b36-b6ea-a8bfe774bd11@grasehotspot.org>
UTC Datetime: 2017-04-17 08:39:48 UTC
Raw Date: Mon, 17 Apr 2017 08:39:48 -0700

Raw message 

Hi 

This is of interest to me also, I have the same issue from time to time.

What i did is to ignore all that lines in the log, but keep all the other 
ones, by creating a new file called

/etc/rsyslog.d/01-chilli.conf

with only this line:

:msg, contains, "(Resource temporarily unavailable)" ~

After that any syslog request log, with that string just gets ignored and 
not logged to file, which is better than your /dev/null temporary solution.

You can still inspect syslog and try to catch the problem.

(waiting for more feedback on this)




sábado, 8 de Abril de 2017 às 13:18:32 UTC+1, Michael Raynor escreveu:
>
> Hi Tim,
>
> Thanks for the reply.
>
> Details are in https://paste.grasehotspot.org/view/4ed0b655
>
> I did reboot the server to add additional compute and memoy earlier in the 
> day but kep /var/log/syslog pointing to /dev/null (as a precaution.
>
> I have removed the symlink and have fixed the permissions and those log 
> entries are no longer showing.
>
> It seems to "go crazy" when there are a huge number of 
> connections/concurrent connections - the peak busy time.
>
> I have also added the follow line to /etc/sysctl.conf to see if it was an 
> issue with empemeral ports (however there was already ~32K outbound ports 
> in there before the change
> net.ipv4.ip_local_port_range = 5000 65535
>
> Let me know if you need further details. I will leave the logging as 
> normal overnight.
>
> Cheers,
>
> Michael
>
> On Saturday, 8 April 2017 20:24:47 UTC+10, timwhite88 wrote:
>>
>> Does a reboot fix it?
>>
>> What version of Coova-chilli is that running? (
>> https://grasehotspot.org/support/collecting-support-information/)
>>
>> Regards
>>
>> On Sat, Apr 8, 2017 at 7:34 AM, Michael Raynor <mx***.@gmail.com> wrote:
>>
>>> Hi Tim,
>>>
>>> We have four big conferences happening back to back over the next four 
>>> weeks.
>>>
>>> I received an alert from New Relic telling me that the grase servers' 
>>> disk was full.
>>>
>>> Having a look I saw that /var/log/syslog was 79GB and was full of the 
>>> following messages - 
>>>
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 806: 11 (Resource 
>>> temporarily unavailable) net_read_eth(fd=5, len=65497, mtu=1500) == -1
>>> Apr  8 01:51:48 grase01 coova-chilli[1382]: net.c: 780: 11 (Resource 
>>> temporarily unavailable) could not read packet
>>>
>>> The syslog file is still growing as I write this and I am afraid it will 
>>> run out again very soon.
>>>
>>> Thanks for your help!
>>>
>>> Michael
>>>
>>> -- 
>>> This mailing list is for the Grase Hotspot Project 
>>> http://grasehotspot.org
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Grase Hotspot" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to gr***.@grasehotspot.org.
>>> To post to this group, send email to gr***.@grasehotspot.org.
>>> Visit this group at 
>>> https://groups.google.com/a/grasehotspot.org/group/grase-hotspot/.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/acff143d-0233-476e-a254-e795559735b2%40grasehotspot.org 
>>> <https://groups.google.com/a/grasehotspot.org/d/msgid/grase-hotspot/acff143d-0233-476e-a254-e795559735b2%40grasehotspot.org?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>

Thread