Uploaded image for project: 'Crunch (Retired)'
  1. Crunch (Retired)
  2. CRUNCH-626

HBaseSourceTarget Binary compatibility across HBase versions

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: IO
    • Labels:
      None

      Description

      Currently Crunch targets HBase 1.0. Certain vendors however are trying to maintain API compatibility with HBase 0.96, 0.98, 1.0, and all 1.x.

      While the API compatibility is ok across these versions, it was however not binary compatible as HBase changed the method signature from returning void[1] to returning OperationWithAttributes[1] (and in later versions Scan).

      This results in an exception like the following, when you try to use the latest Crunch 0.14 release against that vendor's distribution of HBase which we should be theoretically compatible with.

      java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/Scan;
      at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:112)
      at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:96)
      at org.apache.crunch.io.hbase.HBaseSourceTarget.<init>(HBaseSourceTarget.java:88)
      

      [1] - https://github.com/apache/hbase/blob/rel/0.98.0/hbase-client/src/main/java/org/apache/hadoop/hbase/client/OperationWithAttributes.java#L40
      [2] - https://github.com/apache/hbase/blob/rel/1.0.0/hbase-client/src/main/java/org/apache/hadoop/hbase/client/OperationWithAttributes.java#L41

        Attachments

        1. CRUNCH-626.patch
          2 kB
          Micah Whitacre

          Activity

            People

            • Assignee:
              mkwhitacre Micah Whitacre
              Reporter:
              mkwhitacre Micah Whitacre
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: