Question

我正在和spark和hbase进行交流。我使用了HBaseTest.scala。

rdd.count（）给出了准确的结果。但是当我尝试做rdd.collect（）时出现以下错误：

java.io.NotSerializableException: org.apache.hadoop.hbase.io.ImmutableBytesWritable
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
    at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
    at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:42)
    at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:73)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:210)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

无法弄清楚问题。我想打印几行hbase表。

Answer 1

我遇到了同样的问题并找到了解决方案here。

以下是代码段

MyClass ex = MyClassFactory.init(1, "value"); // <-- MyClass now is abstract
  // so this is illegal

hBaseRDD.collect（）给出错误

1 个答案: