我安装了OOZIE并且可以运行它,但我在OOZIE和我的项目中遇到了另一个问题:Java代码工作正常,但是当我尝试在Oozie中实现它时,它就不再起作用了 . Oozie中我的 Map 阶段输出规格有问题,但我不知道是什么 .
我的jar文件(名称wordcount1.jar)包括:
--JoinMapper $ MovieMapper
--JoinMapper $ FigureMapper
--JoinReducer
这是我的错误日志:
2016-03-09 01:24:30,441 INFO CallbackServlet:539 - USER [ - ] GROUP [ - ] TOKEN [ - ] APP [ - ] JOB [0000017-160308222246102-oozie-oozi-W]行动[0000017-160308222246102-oozie -oozi-W @ wordcount1]回调行动[0000017-160308222246102-oozie-oozi-W @ wordcount1]
2016-03-09 01:24:30,812 INFO MapReduceActionExecutor:539 - USER [training] GROUP [ - ] TOKEN [] APP [example-workflow] JOB [0000017-160308222246102-oozie-oozi-W]行动[0000017-160308222246102- oozie-oozi-W @ wordcount1]行动完成,外部身份证[job_201603082221_0037]
2016-03-09 01:24:30,912 WARN MapReduceActionExecutor:542 - USER [training] GROUP [ - ] TOKEN [] APP [example-workflow] JOB [0000017-160308222246102-oozie-oozi-W]行动[0000017-160308222246102- oozie-oozi-W @ wordcount1] Launcher ERROR,原因:主类[org.apache.oozie.action.hadoop.MapReduceMain],main()抛出异常,null
2016-03-09 01:24:31,120 INFO ActionEndXCommand:539 - USER [training] GROUP [ - ] TOKEN [] APP [example-workflow] JOB [0000017-160308222246102-oozie-oozi-W]行动[0000017-160308222246102- oozie-oozi-W @ wordcount1]对于SLA,ERROR被视为失败
2016-03-09 01:24:31,250 INFO ActionStartXCommand:539 - USER [training] GROUP [ - ] TOKEN [] APP [example-workflow] JOB [0000017-160308222246102-oozie-oozi-W] ..
我有以下job.properties文件:
nameNode=hdfs://localhost:8020
jobTracker=localhost:8021
queueName=default
examplesRoot=joinoozie
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/user/training/${examplesRoot}
inputDir1=movies1
inputDir2=figures1
我的workflow.xml文件如下:<workflow-app name="example-workflow" xmlns="uri:oozie:workflow:0.4"> <start to="wordcount1" /> <!-- WordCount MapReduce job --> <action name="wordcount1"> <map-reduce> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <prepare> </prepare> <configuration> <property> <name>mapred.mapper.new-api</name> <value>true</value> </property> <property> <name>mapred.reducer.new-api</name> <value>true</value> </property> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> <property> <name>mapreduce.inputformat.class</name> <value>org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat</value> </property> <property> <name>mapreduce.map.class</name> <value>org.apache.hadoop.mapreduce.lib.input.DelegatingMapper</value> </property> <property> <name>mapreduce.input.multipleinputs.dir.formats</name> <value>${inputDir1};org.apache.hadoop.mapred.TextInputFormat,${inputDir2};org.apache.hadoop.mapred.TextInputFormat</value> </property> <property> <name>mapreduce.input.multipleinputs.dir.mappers</name> <value>${inputDir1};JoinMapper$MovieMapper,${inputDir2};JoinMapper$FigureMapper</value> </property> <property> <name>mapreduce.reduce.class</name> <value>JoinReducer</value> </property> <property> <name>mapred.output.dir</name> <value>output_oozie</value> </property> </configuration> </map-reduce> <ok to="end" /> <error to='end'/> </action> <end name="end" /> </workflow-app>
有人可以帮我解决这个问题吗?会很好..
谢谢,安迪
1 回答
Oozie提供了与工作流中不同操作类型相对应的默认属性集 . 您可以阅读有关位置和属性设置顺序的更多信息here