首页 文章

在HBASE中使用协处理器时出现NullPointerException?

提问于
浏览
0

我正在使用带有HDFS的HBASE 0.94.8 . 我已经实现了协处理器来进行值的求和 . 该表只有两行

hbase(main):043:0>扫描'demo'ROTH COLUMN CELL row1 column = info:category,timestamp = 1375438808010,value = web row1 column = info:hits,timestamp = 1375438797824,value = 123 row2 column = info: category,timestamp = 1375438834518,value = mail row2 column = info:hits,timestamp = 1375438822093,value = 1321

hbase(main):043:0>描述'demo'

'demo',{METHOD =>'table_att',coprocessor $ 1 =>'| org.apache.hadoop.hbase.coprocess true or.AggregateImplementation ||'},{NAME =>'info',DATA_BLOCK_ENCODING =>'NONE ',BLOO MFILTER =>'NONE',REPLICATION_SCOPE =>'0',VERSIONS =>'3',COMPRESSION =>'NONE',MIN_VERSIONS =>'0',TTL =>'2147483647',KEEP_DELETED_CELLS =>'false ',BLOCKSIZE =>'65536',IN_MEMORY =>'false',ENCODE_ON_DISK =>'true',BLOCKCACHE =>'true'} 1行(0.0670秒)

我的代码如下:

import org.apache.hadoop.conf.Configuration; 
import org.apache.hadoop.hbase.HBaseConfiguration; 
import org.apache.hadoop.hbase.HColumnDescriptor; 
import org.apache.hadoop.hbase.HTableDescriptor; 
import org.apache.hadoop.hbase.KeyValue; 
import org.apache.hadoop.hbase.client.HBaseAdmin; 
import org.apache.hadoop.hbase.client.HTable; 
import org.apache.hadoop.hbase.client.Result; 
import org.apache.hadoop.hbase.client.ResultScanner; 
import org.apache.hadoop.hbase.client.Scan; 
import org.apache.hadoop.hbase.client.coprocessor.AggregationClient; 
import org.apache.hadoop.hbase.client.coprocessor.LongColumnInterpreter;
import org.apache.hadoop.hbase.util.Bytes; 
import org.apache.hadoop.hbase.coprocessor.ColumnInterpreter; 
import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;

public class webAggregator {

// private static final byte [] EDRP_FAMILY = Bytes.toBytes(“EDRP”);
// private static final byte [] EDRP_QUALIFIER = Bytes.toBytes(“advanceKWh”);
public static void testSumWithValidRange(Configuration conf,
String [] otherArgs)抛出Throwable {
byte [] EDRP_TABLE = Bytes.toBytes(otherArgs [0]);
byte [] EDRP_FAMILY = Bytes.toBytes(otherArgs [1]);
byte [] EDRP_QUALIFIER = Bytes.toBytes(otherArgs [2]);

conf.set(“hbase.zookeeper.quorum”,“master”);
conf.set(“hbase.zookeeper.property.clientPort”,“2222”);

conf.setLong(“hbase.rpc.timeout”,600000);

conf.setLong(“hbase.client.scanner.caching”,1000);
conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
“org.apache.hadoop.hbase.coprocessor.AggregateImplementation”);

// Utility.CreateHBaseTable(conf,otherArgs [1],otherArgs [2],true);
/ * HBaseAdmin admin = new HBaseAdmin(conf);
HTableDescriptor desc = new HTableDescriptor(EDRP_TABLE);
desc.addFamily(new HColumnDescriptor(EDRP_FAMILY));
admin.createTable(降序); * /

AggregationClient aClient = new AggregationClient(conf);
扫描扫描=新扫描();
scan.addColumn(EDRP_FAMILY,EDRP_QUALIFIER);

HTable table = new HTable(conf,“demo”);
Scan s = new Scan();
ResultScanner ss = table.getScanner(s);
for(Result r:ss){
for(KeyValue kv:r.raw()){
System.out.print(new String(kv.getRow())“”);
System.out.print(new String(kv.getFamily())“:”);
System.out.print(new String(kv.getQualifier())“”);
System.out.print(kv.getTimestamp()“”);
System.out.println(new String(kv.getValue()));
}
}

final ColumnInterpreter <Long,Long> ci = new LongColumnInterpreter();
long sum = aClient.sum(Bytes.toBytes(otherArgs [0]),ci,scan);
的System.out.println(总和);
}

/ **
*主要入口点 .
*

  • @param argsThe
    *命令行参数 .
  • @throws Exception
    *运行作业失败时 .
  • /
    public static void main(String [] args)throws Exception {
    配置conf = HBaseConfiguration.create();
    String [] otherArgs = {“demo”,“info”,“hits”};
    尝试{
    testSumWithValidRange(conf,otherArgs);
    } catch(Throwable e){
    e.printStackTrace();
    }
    }}

我的堆栈跟踪如下:

webAggregator.main上的webAggregator.testSumWithValidRange(webAggregator.java:62)中的java.lang.NullPointerException(webAggregator.java:79)

请帮忙 .

2 回答

  • 0

    我和你有同样的错误 . 经过一番调查,我发现问题是我的列类型是整数,所以LongColumnInterpreter.getValue方法返回null .

    从您的代码和结果中,我确信您的'info:hits'列是字符串列,但不是长列 .

    只考虑将命中更改为真正的长列,从hbase shell看它的值应该是这样的

    11Ak8Z4Mswtk00:MXf1NZ                        column=f1:dp, timestamp=1400144073173, value=\x00\x00\x00\x00\x00\x00\x00b
    

    或者您可以自己编写ColumnInterpreter来处理字符串值和 .

  • 0

    问题是关于数据类型 .

    A. put.addColumn(Bytes.toBytes("objects"), Bytes.toBytes("info"), Bytes.toBytes(1.0));
    

    max/min/sum 没问题 . 但

    B. put.addColumn(Bytes.toBytes("objects"), Bytes.toBytes("info"), Bytes.toBytes("1.0"));
    

    不行 . 从hbase-shell,我们可以看到

    A. column=objects:info, timestamp=1525942759312, value=?\xF0\x00\x00\x00\x00\x00\x00
    B. column=objects:info, timestamp=1525941054901, value=1.0
    

相关问题