首页 文章

完成33%后猪 Map 减少工作失败

提问于
浏览
0

我正在apache pig中运行group by子句,它正在创建map reduce工作,它在1/3完成后失败 . 无论如何我可以解决这个问题,因为日志没有给出任何失败的原因 . 我正在寻找以下任何一个 .
1.找到确切错误的一些方法(即内存错误,数据类型错误等)
2.任何使日志更加冗长以在屏幕上写入更多错误消息的方法 .

2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1459707650021_0003
2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a,b
2016-04-03 22:59:40,252 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4],b[2,4] C:  R:
2016-04-03 22:59:40,258 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2016-04-03 22:59:40,258 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:10,428 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 33% complete
2016-04-03 23:00:10,428 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Running jobs are [job_1459707650021_0003]
2016-04-03 23:00:15,435 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2016-04-03 23:00:15,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_1459707650021_0003 has failed! Stop running all dependent jobs
2016-04-03 23:00:15,435 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2016-04-03 23:00:15,508 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://compute.internal:8188/ws/v1/timeline/
2016-04-03 23:00:15,511 [main] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at compute.internal/8050
2016-04-03 23:00:15,515 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=FAILED. Redirecting to job history server

1 回答

  • 0

    JobId与您提交给map reduce框架的每个作业相关联 . 复制该JobId,粘贴并在您的工作跟踪器中搜索它 . 例如,在浏览器中的群集cloudera的CDH3中,他们设置了作业跟踪器,您可以在其中查看所有作业的详细信息 .

相关问题