beyond virtual memory limits

查看錯誤日志

不一定能在 Client日志 里面看到錯誤,可以打開AM的WEB頁面查看:

Application
appattempt
Diagnostics Info:   
AM Container for appattempt_1465808152295_0002_000001 exited with exitCode: -103
For more detailed output, check application tracking page:http://bigdatamgr1:8088/proxy/application_1465808152295_0002/Then, click on links to logs of each attempt.
Diagnostics: Container [pid=68226,containerID=container_1465808152295_0002_01_000001] is running beyond virtual memory limits. Current usage: 346.6 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container. 

查找解決辦法

Google: 【2.5 GB of 2.1 GB virtual memory used. Killing container.】

http://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits

http://stackoverflow.com/questions/14110428/am-container-is-running-beyond-virtual-memory-limits

https://issues.apache.org/jira/browse/YARN-4714

也可以調(diào)整opts和memory兩個配置。

修改配置

[eshore@bigdatamgr1 ~]$ cat hadoop/etc/hadoop/yarn-site.xml 
...
<property>
  <name>yarn.nodemanager.vmem-check-enabled</name>
  <value>false</value>
  <description>Whether virtual memory limits will be enforced for containers</description>
</property>
<property>
  <name>yarn.nodemanager.vmem-pmem-ratio</name>
  <value>4</value>
  <description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
</property>

同步到所有nodemanager,然后重啟yarn。

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

推薦閱讀更多精彩內(nèi)容