Discussion Forums
Discussion Forums > Category: Compute > Forum: Amazon Elastic Compute Cloud (EC2) >Thread: EC2 instance hangs (dhclient: fork: Cannot allocate memory)
Advanced search options
EC2 instance hangs (dhclient: fork: Cannot allocate memory)
Posted by: dhalatsis
Posted on: Sep 7, 2015 5:55 AM
  Click to reply to this thread Reply
nodejs , dhclient , freeze , error , memory , fork
This question is not answered. Answer it to earn points.
Hi guys. I am running a t2.micro Ubuntu instance as webserver running a node.js application behind a nginx proxy. My problem is that suddenly the server freezed and I couldn't even access through ssh. The only option was to restart the instance through the management console.

In my node's script logs I cannot see any errors, except for the fact that script stopped operating at a specific time. At the same time in the system logs of the instance, I can see that there was an out off memory error during dhclient operation. More specifically:
Sep  5 06:17:01 ip-172-31-32-75 CRON[8622]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Sep  5 06:20:24 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 06:20:24 ip-172-31-32-75 dhclient: DHCPACK of 172.31.32.75 from 172.31.32.1
Sep  5 06:20:24 ip-172-31-32-75 dhclient: bound to 172.31.32.75 -- renewal in 1794 seconds.
Sep  5 06:25:01 ip-172-31-32-75 CRON[8638]: (root) CMD (test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ))
Sep  5 06:39:01 ip-172-31-32-75 cron[1110]: (CRON) error (can't fork)
Sep  5 06:50:22 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 06:50:33 ip-172-31-32-75 dhclient: DHCPACK of 172.31.32.75 from 172.31.32.1
Sep  5 06:51:00 ip-172-31-32-75 dhclient: fork: Cannot allocate memory
Sep  5 06:51:15 ip-172-31-32-75 dhclient: bound to 172.31.32.75 -- renewal in 1417 seconds.
Sep  5 07:17:14 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 07:18:39 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 07:20:52 ip-172-31-32-75 dhclient: DHCPACK of 172.31.32.75 from 172.31.32.1
Sep  5 07:28:14 ip-172-31-32-75 dhclient: fork: Cannot allocate memory
Sep  5 07:32:09 ip-172-31-32-75 dhclient: bound to 172.31.32.75 -- renewal in 890 seconds.
Sep  5 07:47:53 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 07:48:57 ip-172-31-32-75 dhclient: DHCPREQUEST of 172.31.32.75 on eth0 to 172.31.32.1 port 67 (xid=0x814a48d)
Sep  5 07:50:53 ip-172-31-32-75 dhclient: DHCPACK of 172.31.32.75 from 172.31.32.1

Note the error at 06:35
also in the cloud watch metrics I see a rise in CPU utilization from 2-3 percent to 12% that stays there for half an hour......

Does anybody have an idea of what to look in order to determine the cause of the freeze????
Thank you for your time

Edited by: dhalatsis on Sep 7, 2015 3:55 PM

Edited by: dhalatsis on Sep 7, 2015 6:06 PM
Permlink Replies: 1 | Pages: 1 - Last Post: Oct 12, 2017 1:35 AM by: MR M MCGUINNESS
Replies
Re: EC2 instance hangs (dhclient: fork: Cannot allocate memory)
Posted by: MR M MCGUINNESS
Posted on: Oct 12, 2017 1:35 AM
in response to: dhalatsis in response to: dhalatsis
  Click to reply to this thread Reply
Did you managed to get to the bottom of this or find a workaround?

I have exactly the same thing with a t2.micro running Apache, PHP and MySQL. It is only hosting a couple of very low traffic WordPress sites but every day or two the whole machine becomes unresponsive because of the "dhclient: fork: Cannot allocate memory" problem.

At that point you can't ssh to the machine even so you need to stop/start the instance from AWS.