Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17989

eclipse issues with compiling scala code in hbase-spark

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build, spark
    • Labels:
      None
    • Environment:

      Description

      This is very possibly an issue with my local environment, but it seems strange enough that I'm not sure.

      There are tons of compilation problems in the hbase-spark module. I first noticed them because eclipse was complaining very loudly, but was very confused when mvn package seemed to succeed. In trying to investigate, I took a look at the generated class file and saw what looks like javac error output:

      $ less hbase-spark/target/classes/org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter.class
      
      <CA><FE><BA><BE>^@^@^@4^@R^G^@^B^A^@4org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter^G^@^D^A^@)org/apache/hadoop/hbase/filter/FilterBase^A^@^Clog^A^@ Lorg/apache/commons/logging/Log;^A^@^VdynamicLogicExpression^A^@^XLDynamicLogicExpression;^A^@^SvalueFromQueryArray^A^@^C[[B^A^@ESCcurrentCellToColumnIndexMap^A^@^SLjava/util/HashMap;^A^@  Signature^A^@fLjava/util/HashMap<LByteArrayComparable;Ljava/util/HashMap<LByteArrayComparable;Ljava/lang/String;>;>;^A^@^ZcolumnToCurrentRowValueMap^A^@<Ljava/util/HashMap<Ljava/lang/String;LByteArrayComparable;>;^A^@^LrowKeyFamily^A^@^B[B^A^@^OrowKeyQualifier^A^@^PencoderClassName^A^@^RLjava/lang/String;^A^@^F<init>^A^@C(LDynamicLogicExpression;[[BLjava/util/HashMap;Ljava/lang/String;)V^A^@<96>(LDynamicLogicExpression;[[BLjava/util/HashMap<LByteArrayComparable;Ljava/util/HashMap<LByteArrayComparable;Ljava/lang/String;>;>;Ljava/lang/String;)V^A^@^DCode^G^@ESC^A^@^Ojava/lang/Erro^@^]^A^M"Unresolved compilation problems: 
              The import org.apache.hadoop.hbase.spark.datasources.BytesEncoder cannot be resolved
              The import org.apache.hadoop.hbase.spark.datasources.JavaBytesEncoder cannot be resolved
              The import org.apache.spark.sql.datasources.hbase.Field cannot be resolved
              DynamicLogicExpression cannot be resolved to a type
              ByteArrayComparable cannot be resolved to a type
              ByteArrayComparable cannot be resolved to a type
              ByteArrayComparable cannot be resolved to a type
              DynamicLogicExpression cannot be resolved to a type
      <snip>
      

      This problem is present in several classes.

      $ grep -lrF 'Unresolved compilation problem' **/*.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkDeleteExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkGetExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkLoadExample$BulkLoadFunction.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkLoadExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkPutExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseDistributedScan.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseMapGetPutExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseStreamingBulkPutExample.class
      hbase-spark/target/classes/org/apache/hadoop/hbase/spark/SparkSQLPushDownFilter.class
      hbase-spark/target/test-classes/org/apache/hadoop/hbase/spark/TestJavaHBaseContext$BulkLoadFunction.class
      hbase-spark/target/test-classes/org/apache/hadoop/hbase/spark/TestJavaHBaseContext$BulkLoadThinRowsFunction.class
      hbase-spark/target/test-classes/org/apache/hadoop/hbase/spark/TestJavaHBaseContext.class
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mdrob Mike Drob
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: