Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None

      Description

      Pig doesn't support HBase 2.0.0. Since the new HBase API introduces several API changes, we should find a way to support both 1.x and 2.x HBase API.

      1. PIG-5191_1.patch
        34 kB
        Nandor Kollar

        Activity

        Hide
        nkollar Nandor Kollar added a comment -

        Either we can solve this via shims (like we had shims for different Hadoop version), or we can use reflection. Rohini Palaniswamy what do you think, which one would you prefer? Do you have any other idea for supporting two different HBase APIs? I'd prefer shims, because I think the code gets cleaner, not overwhelmed with reflection code and with reflection we'll get exception at runtime, if incorrect HBase version is used.

        Show
        nkollar Nandor Kollar added a comment - Either we can solve this via shims (like we had shims for different Hadoop version), or we can use reflection. Rohini Palaniswamy what do you think, which one would you prefer? Do you have any other idea for supporting two different HBase APIs? I'd prefer shims, because I think the code gets cleaner, not overwhelmed with reflection code and with reflection we'll get exception at runtime, if incorrect HBase version is used.
        Hide
        rohini Rohini Palaniswamy added a comment -

        Reflection is preferable if there are just one or two incompatibilities as one does not have to compile twice. If there is lot of incompatibilities shims is the way to go as it will be more cleaner.

        Show
        rohini Rohini Palaniswamy added a comment - Reflection is preferable if there are just one or two incompatibilities as one does not have to compile twice. If there is lot of incompatibilities shims is the way to go as it will be more cleaner.
        Show
        rohini Rohini Palaniswamy added a comment - Just to update, approach that will be taken will be https://issues.apache.org/jira/browse/PIG-5246?focusedCommentId=16035045&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16035045
        Hide
        nkollar Nandor Kollar added a comment -

        Actually looks like replacing the deprecated API calls is sufficient, using the non-deprecated alternatives is compatible both with HBase 1.x and 2.x APIs.

        Show
        nkollar Nandor Kollar added a comment - Actually looks like replacing the deprecated API calls is sufficient, using the non-deprecated alternatives is compatible both with HBase 1.x and 2.x APIs.
        Hide
        daijy Daniel Dai added a comment -

        Looks good. Can you also try if bin/pig would register all dependent hbase jars automatically in hbase2? ie, no need to manually register jars when using HBaseStorage. We have done this for hbase1.

        Show
        daijy Daniel Dai added a comment - Looks good. Can you also try if bin/pig would register all dependent hbase jars automatically in hbase2? ie, no need to manually register jars when using HBaseStorage. We have done this for hbase1.
        Hide
        nkollar Nandor Kollar added a comment -

        Daniel Dai thanks for looking at my patch, sure, I'll have a look at bin/pig.

        Show
        nkollar Nandor Kollar added a comment - Daniel Dai thanks for looking at my patch, sure, I'll have a look at bin/pig.

          People

          • Assignee:
            nkollar Nandor Kollar
            Reporter:
            nkollar Nandor Kollar
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development