首页 文章

WSO2外部hadoop

提问于
浏览
2

我们希望将我们的WSO2 BAM 2.0.1连接到外部hadoop集群(完全分布式) . 我按照此链接进行设置

Configuring Apache Hive

但是,当我运行我的Hive脚本时,我得到以下错误

java.sql.SQLException:查询返回非零代码:9,原因:FAILED:执行错误,从org.apache.hadoop.hive.ql.exec.MapRedTask返回代码-101

这个Hive脚本适用于本地Hadoop模式......

在使用外部Hadoop集群时,是否还需要对Hive脚本进行任何更改 .

这些是我在控制台中看到的错误..

TID:[0] [BAM] [2013-01-16 09:42:36,583] INFO - 未缓存hdfs的内容摘要:// r4p8n0c:9000 / user / hive / warehouse_wso2-1234 / aeanalyticstable_demo TID:[0] [BAM] [2013-01-16 09:42:36,583] INFO - 未缓存hdfs的内容摘要:// r4p8n0c:9000 / user / hive / warehouse_wso2-1234 / aeanalyticstable_demo TID :[0] [BAM] [2013-01-16 09:42:36,590] INFO - 添加非原生表hdfs:// r4p8n0c:9000 / user / hive / warehouse_wso2-1234 / aeanalyticstable_demo TID:[0] [BAM] [2013-01-16 09:42:36,590] INFO {org.apache . hadoop.hive.ql.exec.ExecDriver} - 添加非原生表hdfs:// r4p8n0c:9000 / user / hive / warehouse_wso2-1234 / aeanalyticstable_demo TID :[0] [BAM] [2013-01-16 09:42:36,832]错误 - 失败:执行错误,返回代码-101来自org.apache.hadoop.hive.ql.exec.MapRedTask TID:[0] [BAM] [2013-01-16 09:42:36,832]错误 - 失败:执行错误,从org.apache.hadoop.hive.ql返回代码-101 . exec.MapRedTask TID:[0] [BAM] [2013-01-16 09:42:36,833] INFO - TID:[0] [BAM] [2013-01-16 09:42:36,833] INFO - TID:[0] [BAM] [2013-01-16 09:42:36,833] INFO - {org . apache.hadoop.hive.ql.Driver} TID:[0] [BAM] [2013-01-16 09:42:36,833] INFO - {org.apache . hadoop.hive.ql.Driver} TID:[0] [BAM] [2013-01-16 09:42:36,834] INFO - {org.apache.hadoop . hive.ql.Driver} TID:[0] [BAM] [2013-01-16 09:42:36,834] INFO {org.apach电子邮件:[0] [BAM] [2013-01-16 09:42:36,834]错误{org.wso2 . carbon.analytics.hive.impl.HiveExecutorServiceImpl} - 执行Hive脚本时出错 . 查询返回非零代码:9,原因:FAILED:执行错误,返回代码-101来自org.apache.hadoop.hive.ql.exec.MapRedTask java .sql.SQLException:查询返回非零代码:9,原因:FAILED:执行错误,从org.apache.hadoop.hive.ql.exec.MapRedTask返回代码-101 atg.apache.hadoop.hive.jdbc . 位于org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl $的org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl $ ScriptCallable.call(HiveExecutorServiceImpl.java:325)中的HiveStatement.executeQuery(HiveStatement.java:189) ScriptCallable.call(HiveExecutorServiceImpl.java:225)位于java.util.concurrent.FutureTask $ Sync.innerRun(FutureTask.java:303),位于java.util的java.util.concurrent.FutureTask.run(FutureTask.java:138) .concurrent.ThreadPoolExecutor $ Worker.runTask(ThreadPoolExecutor.java:886)at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:908)at java.lang.Thread.run(Thread.java:662) TID:[0] [BAM] [2013-01-16 09:42:36,848] ERROR - 执行Hive脚本时出错 . 查询返回非零代码:9,原因:FAILED:执行错误,从org.apache.hadoop.hive.ql.exec.MapRedTask org.wso2返回代码-101 . carbon.analytics.hive.stub.HiveExecutionServiceHiveExecutionException:sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)at HiveExecutionServiceHiveExecutionException atsun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27),位于java.lang.Class.newInstance(Class.java:355)的java.lang.ref.Constructor.newInstance(Constructor.java:513) .glass.newInstance(Class.java:308)org.wso2.carbon.analytics.hive.stub.HiveExecutionServiceStub.executeHiveScript(HiveExecutionServiceStub.java:215)org.wso2.carbon.analytics.hive.ui.client.HiveExecutionClient位于org.apache.jsp.hive_002dexplorer.queryresults_jsp._jspService(queryresults_jsp.java:90)的.executeScript(HiveExecutionClient.java:66)位于javax的org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) . servlet.http.HttpServlet.service(HttpServlet.java:722)org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:332)org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java: 314)atg.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)at javax.servlet.http.HttpServlet.service(HttpServlet.java:7 22)atg.wso2.carbon.ui.JspServlet.service(JspServlet.java:161)at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)at javax.servlet.http.HttpServlet.service (HttpServlet.java:722)org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:36)org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90 )在org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)的org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67)javax . Servlet.http.HttpServlet.service(HttpServlet.java:722)org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( ApplicationFilterChain.java:305)org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)at org.apache.catalina.core.ApplicationDispatcher.invoke (ApplicationDispatcher.java:690)org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:599)

2 回答

  • 0

    如果您已正确设置外部hadoop群集,它应该可以工作 . 它已经使用外部hadoop集群进行了测试 . 如果您遇到任何问题,请检查hadoop集群节点中的错误日志 . 找到与hadoop集群相关的配置问题会很有帮助 . 因为有时在ui中显示的问题不是很有用 . 您无需更改Hive脚本即可使用外部群集 .

    谢谢,卡苏 .

  • 0

    我们能够让WSO2 BAM与外部完全分布式Hadoop配合使用 .

    看起来像2.0.1版本在向外部hadoop集群提交map-reduce作业时遇到麻烦 . 我们支持2.0.0并且Hive Scripts开始工作 .

    我想知道这是2.0.1的已知问题吗?我们有一个2.0.1的精简版(它只有碳,Analytics / AnalyticsUI,仪表板,注册表,Databridge,数据源,stratos,安全,任务和传输管理功能)

    谢谢拉吉夫

相关问题