我有一个简单的Spark程序来从Hbase表中获取数据
final SparkConf sparkConf = new SparkConf()
.setAppName("phoenix-spark")
.set("spark.executor.memory", "2g")
.setMaster(SPARK_MASTER_URL);
JavaSparkContext jsc = new JavaSparkContext(sparkConf);
final Configuration configuration = HBaseConfiguration.create();
configuration.set(HConstants.ZOOKEEPER_QUORUM, ZOOKEEPER_QUORUM_URL);
PhoenixConfigurationUtil.setInputTableName(configuration , "STOCKS");
PhoenixConfigurationUtil.setOutputTableName(configuration , "STOCKS");
PhoenixConfigurationUtil.setInputQuery(configuration, "SELECT STOCK_NAME,RECORDING_YEAR,RECORDINGS_QUARTER FROM STOCKS");
PhoenixConfigurationUtil.setInputClass(configuration, StockWritable.class);
// PhoenixConfigurationUtil.setUpsertColumnNames(configuration,"STOCK_NAME,RECORDING_YEAR,RECORDINGS_AVG");
configuration.setClass(JobContext.OUTPUT_FORMAT_CLASS_ATTR,PhoenixOutputFormat.class, OutputFormat.class);
@SuppressWarnings("unchecked")
JavaPairRDD<NullWritable, StockWritable> stocksRDD = jsc.newAPIHadoopRDD(
configuration,
PhoenixInputFormat.class,
NullWritable.class,
StockWritable.class);
System.out.println(String.format("&&&&&&&&&&&&&&&&&&&&&&&& the number of records are [%s] ", stocksRDD.count()));
我收到以下错误:
线程“main”中的异常java.lang.RuntimeException:java.sql.SQLException:ERROR 103(08004):无法 Build 连接 . 在在org.apache.spark.rdd.NewHadoopRDD.getPartitions org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:203)在org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:88) (NewHadoopRDD.scala:125)在org.apache.spark.rdd.RDD $$ anonfun $分区$ 2.适用(RDD.scala:252)在org.apache.spark.rdd.RDD $$ anonfun $分区$ 2.适用(RDD.scala:250)在scala.Option.getOrElse(Option.scala:121)在org.apache.spark.rdd.RDD.partitions(RDD.scala:250)在org.apache.spark.SparkContext.runJob( SparkContext.scala:2094)org.apache.spark.rdd.RDD.count(RDD.scala:1158)org.apache.spark.api.java.JavaRDDLike $ class.count(JavaRDDLike.scala:455)at org .apache.spark.api.java.AbstractJavaRDDLike.count(JavaRDDLike.scala:45)在PhoenixSparkJob.main(PhoenixSparkJob.java:53)在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)在sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImp l.invoke(DelegatingMethodAccessorImpl.java:43)在java.lang.reflect.Method.invoke(Method.java:498)在org.apache.spark.deploy.SparkSubmit $ .ORG $阿帕奇$火花$部署$ SparkSubmit $$ runMain(SparkSubmit.scala:775)在org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1(SparkSubmit.scala:180)在org.apache.spark.deploy.SparkSubmit $ .submit(SparkSubmit.scala:205)在Org.apache.spark.deploy.SparkSubmit $ .main(SparkSubmit.scala:119)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)