Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18863

spark over hbase snapshot

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2.0
    • Fix Version/s: None
    • Component/s: Client, snapshots, spark
    • Labels:
      None

      Description

      The configuration error in two method. First is ok, and second is wrong.

      1.
      val hConf = HBaseConfiguration.create()
      // ignore some config
      
      val hBaseRDD = sc.newAPIHadoopRDD(hConf,
            classOf[TableSnapshotInputFormat],
            classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],
            classOf[org.apache.hadoop.hbase.client.Result])
      2.
      val hConf = HBaseConfiguration.create()
      val job = Job.getInstance(hConf)
      val hBaseRDD = sc.newAPIHadoopRDD(job.getConfiguration,
            classOf[TableSnapshotInputFormat],
            classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable],
            classOf[org.apache.hadoop.hbase.client.Result])
      
      and the log:
      Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:264)
      at org.apache.hadoop.hbase.io.encoding.DataBlockEncoding.createEncoder(DataBlockEncoding.java:186)
      ... 21 more
      

      I'm sure that I have upload jar with spark cmd -jars.
      Is this a bug?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ulysses ulysses you
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: