Hive
  1. Hive
  2. HIVE-3725

Add support for pulling HBase columns with prefixes

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.12.0
    • Component/s: HBase Handler
    • Labels:

      Description

      Current HBase Hive integration supports reading many values from the same row by specifying a column family. And specifying just the column family can pull in all qualifiers within the family.

      We should add in support to be able to specify a prefix for the qualifier and all columns that start with the prefix would automatically get pulled in. A wildcard support would be ideal.

      1. HIVE-3725.4.patch.txt
        17 kB
        Swarnim Kulkarni
      2. HIVE-3725.3.patch.txt
        17 kB
        Brock Noland
      3. HIVE-3725.patch.3.txt
        17 kB
        Swarnim Kulkarni
      4. HIVE-3725.2.patch.txt
        11 kB
        Swarnim Kulkarni
      5. HIVE-3725.1.patch.txt
        10 kB
        Swarnim Kulkarni

        Activity

        Hide
        Swarnim Kulkarni added a comment -

        Patch to add support for pulling hbase columns with prefixes.

        Show
        Swarnim Kulkarni added a comment - Patch to add support for pulling hbase columns with prefixes.
        Hide
        Swarnim Kulkarni added a comment -

        This patch should now be ready for review.

        Show
        Swarnim Kulkarni added a comment - This patch should now be ready for review.
        Hide
        Swarnim Kulkarni added a comment -
        Show
        Swarnim Kulkarni added a comment - Review request: https://reviews.apache.org/r/9276/
        Hide
        Mark Grover added a comment -

        Comments on reviewboard

        Show
        Mark Grover added a comment - Comments on reviewboard
        Hide
        Swarnim Kulkarni added a comment -

        Patch updated with proposed changes on the review.

        Show
        Swarnim Kulkarni added a comment - Patch updated with proposed changes on the review.
        Hide
        Swarnim Kulkarni added a comment -

        If a committer can take a quick look at the updated patch, I will highly appreciate that. Thanks!

        Show
        Swarnim Kulkarni added a comment - If a committer can take a quick look at the updated patch, I will highly appreciate that. Thanks!
        Hide
        Brock Noland added a comment -

        Unless someone looks sooner I can take a look tomorrow morning (CST).

        Show
        Brock Noland added a comment - Unless someone looks sooner I can take a look tomorrow morning (CST).
        Hide
        Brock Noland added a comment -

        Jeez, sorry I thought Navis commented on this one but I guess that was HIVE-3553. Reviewing now.

        Show
        Brock Noland added a comment - Jeez, sorry I thought Navis commented on this one but I guess that was HIVE-3553 . Reviewing now.
        Hide
        Brock Noland added a comment -

        Feedback provided on RB, cancelling patch.

        Show
        Brock Noland added a comment - Feedback provided on RB, cancelling patch.
        Hide
        Brock Noland added a comment -

        Hi,

        I am re-uploading your patch as "HIVE-3725.3.patch.txt" so the precommit tests will execute.

        Show
        Brock Noland added a comment - Hi, I am re-uploading your patch as " HIVE-3725 .3.patch.txt" so the precommit tests will execute.
        Hide
        Brock Noland added a comment -

        Marking "Patch Available" so the precommit tests will execute.

        Show
        Brock Noland added a comment - Marking "Patch Available" so the precommit tests will execute.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12593448/HIVE-3725.3.patch.txt

        ERROR: -1 due to 1 failed/errored test(s), 2647 tests executed
        Failed tests:

        org.apache.hcatalog.mapreduce.TestHCatInputFormat.testBadRecordHandlingPasses
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/124/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/124/console

        Messages:

        Executing org.apache.hive.ptest.execution.CleanupPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593448/HIVE-3725.3.patch.txt ERROR: -1 due to 1 failed/errored test(s), 2647 tests executed Failed tests: org.apache.hcatalog.mapreduce.TestHCatInputFormat.testBadRecordHandlingPasses Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/124/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/124/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
        Hide
        Swarnim Kulkarni added a comment -

        Looking into the test failure, as per my understanding doesn't seem to be related to this change.

        Show
        Swarnim Kulkarni added a comment - Looking into the test failure, as per my understanding doesn't seem to be related to this change.
        Hide
        Brock Noland added a comment -

        This looks pretty good to me. I am just going to run the precommit tests again. If you have a chance I think we should add a javadoc to HBASE_COLUMNS_REGEX_MATCHING saying we don't so full regex matching at present. Not a huge deal I could add that on commit as well.

        Show
        Brock Noland added a comment - This looks pretty good to me. I am just going to run the precommit tests again. If you have a chance I think we should add a javadoc to HBASE_COLUMNS_REGEX_MATCHING saying we don't so full regex matching at present. Not a huge deal I could add that on commit as well.
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12593448/HIVE-3725.3.patch.txt

        SUCCESS: +1 2647 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/127/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/127/console

        Messages:

        Executing org.apache.hive.ptest.execution.CleanupPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593448/HIVE-3725.3.patch.txt SUCCESS: +1 2647 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/127/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/127/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Hide
        Swarnim Kulkarni added a comment -

        Added the javadoc comment.

        Show
        Swarnim Kulkarni added a comment - Added the javadoc comment.
        Hide
        Brock Noland added a comment -

        +1

        Show
        Brock Noland added a comment - +1
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12593553/HIVE-3725.4.patch.txt

        SUCCESS: +1 2647 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/129/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/129/console

        Messages:

        Executing org.apache.hive.ptest.execution.CleanupPhase
        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12593553/HIVE-3725.4.patch.txt SUCCESS: +1 2647 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/129/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/129/console Messages: Executing org.apache.hive.ptest.execution.CleanupPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Hide
        Brock Noland added a comment -

        Thank you for your contribution! I committed this to trunk!

        Show
        Brock Noland added a comment - Thank you for your contribution! I committed this to trunk!
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          People

          • Assignee:
            Swarnim Kulkarni
            Reporter:
            Swarnim Kulkarni
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development