首页 文章

hadoop nodemanager无法启动,slave不满足最小分配

提问于
浏览
1

运行start-all.sh时,我的slave1和slave2在jps中找不到nodemanager

使用VM virtualbox master在Ubuntu 16.04上工作

yunchi@master:~$ jps
15920 Jps
15505 SecondaryNameNode
15659 ResourceManager
15293 NameNode

SLAVE1

yunchi@slave1:~$ jps
7587 DataNode
7945 Jps

SLAVE2

yunchi@slave2:~$ jps
7138 DataNode
7374 Jps

这是我的nodemanager-slave1.log

2017年10月12日17:26:59662 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl:从ResourceManager中,节点管理器登记收到关机信号:意外的错误起始NodeStatusUpdater org.apache.hadoop.yarn.exceptions.YarnRuntimeException来自ResourceManager的消息:来自slave1的NodeManager不满足最小分配,将SHUTDOWN信号发送到NodeManager . 在org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)在org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)在org.apache .hadoop.service.AbstractService.start(AbstractService.java:193)在org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)在org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart (NodeManager.java:272)在org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)在org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:496)在org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:543)

2017年10月12日17:26:59710致命org.apache.hadoop.yarn.server.nodemanager.NodeManager:错误启动节点管理器org.apache.hadoop.yarn.exceptions.YarnRuntimeException:org.apache.hadoop.yarn.exceptions . YarnRuntimeException:从Resourcemanager接收SHUTDOWN信号,NodeManager注册失败,来自ResourceManager的消息:来自slave1的NodeManager不满足最小分配,向NodeManager发送SHUTDOWN信号 . 在org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:203)在org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)在org.apache.hadoop.service .gositeService.serviceStart(CompositeService.java:120)atg.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:272)org.apache.hadoop.service.AbstractService.start(AbstractService.java) :193)在org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:496)在org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:543)引起by:org.apache.hadoop.yarn.exceptions.YarnRuntimeException:从Resourcemanager收到SHUTDOWN信号,NodeManager注册失败,来自ResourceManager的消息:来自slave1的NodeManager不满足最小分配,向NodeManager发送SHUTDOWN信号 . 在org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)在org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)... 6更多

我找到了一些相关信息 . 他们告诉我补充一下

<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>1024</value>
</property>

但它仍然不起作用 .

所以,我设定值4096,仍然无法正常工作

这是我下面的yarn-site.xml

<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>

<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>

<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>

<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>

<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>

<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
</property>

<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/home/yunchi/yarn</value>
<final>true</final>
</property>


<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-vcores</name>
<value>1</value>
</property>

1 回答

  • 0

    也许您的虚拟机没有足够的资源 . 所以你不应该增加参数 yarn.nodemanager.resource.memory-mb 的值,因为没有资源 . 尝试为此参数设置较小的值,例如256 MB .

    检查这个post .

相关问题