首页 文章

Oozie map-reduce工作永远停留在PREP状态

提问于
浏览
3

我正在尝试运行wordcount map-reduce示例,引用https://support.pivotal.io/hc/en-us/articles/203355837-How-to-run-a-Map-Reduce-jar-using-Oozie-workflow .

我在群集设置中使用hadoop 2.2.0

问题是我的oozie map-reduce工作继续运行 . 它没有给出任何错误但是没有成功完成 . 没有错误消息 . 也没有创建输出目录(假设发生) .

以下是workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.1" name="map-reduce-wf">
 <start to="mr-node"/>
 <action name="mr-node">
     <map-reduce>
       <job-tracker>${jobTracker}</job-tracker>
       <name-node>${nameNode}</name-node>
       <prepare>
         <delete path="${nameNode}/user/hadoop/${examplesRoot}/output-data/${outputDir}"/>
       </prepare>

   <configuration>
     <property>
       <name>mapred.mapper.new-api</name>
       <value>true</value>
     </property>
     <property>
       <name>mapred.reducer.new-api</name>
       <value>true</value>
     </property>
     <property>
       <name>mapred.job.queue.name</name>
       <value>${queueName}</value>
     </property>
     <property>
       <name>mapreduce.map.class</name>
       <value>org.apache.hadoop.examples.WordCount$TokenizerMapper</value>
     </property>
     <property>
       <name>mapreduce.reduce.class</name>
       <value>org.apache.hadoop.examples.WordCount$IntSumReducer</value>
     </property>
     <property>
       <name>mapreduce.combine.class</name>
       <value>org.apache.hadoop.examples.WordCount$IntSumReducer</value>
     </property>
     <property>
       <name>mapred.output.key.class</name>
       <value>org.apache.hadoop.io.Text</value>
     </property>
     <property>
       <name>mapred.output.value.class</name>
       <value>org.apache.hadoop.io.IntWritable</value>
     </property>
     <property>
       <name>mapred.input.dir</name>
       <value>/user/hadoop/${examplesRoot}/input-data/text</value>
     </property>
     <property>
       <name>mapred.output.dir</name>
       <value>/user/hadoop/${examplesRoot}/output-data/${outputDir}</value>
     </property>
   </configuration>
  </map-reduce>
  <ok to="end"/>
  <error to="fail"/>
 </action>
   <kill name="fail">
   <message>Map/Reduce failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
   </kill>
   <end name="end"/>
</workflow-app>

Job.properties

nameNode=hdfs://hadoopmaster:54310
jobTracker=localhost:54311
queueName=default
examplesRoot=examplesoozie

oozie.wf.application.path=${nameNode}/user/hadoop/${examplesRoot}/map-reduce
outputDir=map-reduce

作业状态的输出是

Workflow Name : map-reduce-wf
App Path      : hdfs://hadoopmaster:54310/user/hadoop/examplesoozie/map-reduce
Status        : RUNNING
Run           : 0
User          : hadoopuser
Group         : -
Created       : 2014-10-26 19:52 GMT
Started       : 2014-10-26 19:52 GMT
Last Modified : 2014-10-26 19:52 GMT
Ended         : -
CoordAction ID: -

Actions
------------------------------------------------------------------------------------------------------------------------------------
ID                                                                            Status    Ext ID                 Ext Status Err Code 
------------------------------------------------------------------------------------------------------------------------------------
0000006-141026135258981-oozie-hado-W@:start:                                  OK        -                      OK         -        
------------------------------------------------------------------------------------------------------------------------------------
0000006-141026135258981-oozie-hado-W@mr-node                                  PREP      -                      -          -        
------------------------------------------------------------------------------------------------------------------------------------

任何帮助将不胜感激 .

1 回答

  • 0

    在job.properties中,initialDataset应该在jobStart之前8小时(PST是GMT-8) . 你已经为两者设定了格林威治标准时间2014-10-26 19:52 .

相关问题