Hive
  1. Hive
  2. HIVE-2898

Add nicer helper functions for adding and reading metadata from RCFiles

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Labels:
      None

      Description

      Currently, to use the metadata in RCFile, you need to manipulate it using SequenceFile.Metadata. I'd like to add two helper functions that make it more convenient.

        Activity

        Hide
        Phabricator added a comment -

        omalley requested code review of "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA

        hive-2898

        add helper functions for RCFile metadata

        Currently, to use the metadata in RCFile, you need to manipulate it using SequenceFile.Metadata. I'd like to add two helper functions that make it more convenient.

        TEST PLAN
        add tests to test the new function calls

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/5469/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - omalley requested code review of " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA hive-2898 add helper functions for RCFile metadata Currently, to use the metadata in RCFile, you need to manipulate it using SequenceFile.Metadata. I'd like to add two helper functions that make it more convenient. TEST PLAN add tests to test the new function calls REVISION DETAIL https://reviews.facebook.net/D2433 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5469/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Phabricator added a comment -

        omalley requested code review of "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA

        <<Enter Revision Title>>

        fixed syntax error in test code

        Currently, to use the metadata in RCFile, you need to manipulate it using SequenceFile.Metadata. I'd like to add two helper functions that make it more convenient.

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D2439

        AFFECTED FILES
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/5475/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - omalley requested code review of " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA <<Enter Revision Title>> fixed syntax error in test code Currently, to use the metadata in RCFile, you need to manipulate it using SequenceFile.Metadata. I'd like to add two helper functions that make it more convenient. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2439 AFFECTED FILES ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5475/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Phabricator added a comment -

        omalley has abandoned the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".

        This should have merged with the previous one.

        REVISION DETAIL
        https://reviews.facebook.net/D2439

        Show
        Phabricator added a comment - omalley has abandoned the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". This should have merged with the previous one. REVISION DETAIL https://reviews.facebook.net/D2439
        Hide
        Phabricator added a comment -

        omalley updated the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA

        fixed syntax error in test

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        Show
        Phabricator added a comment - omalley updated the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA fixed syntax error in test REVISION DETAIL https://reviews.facebook.net/D2433 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Hide
        Phabricator added a comment -

        omalley updated the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA

        fixed a merge problem

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        Show
        Phabricator added a comment - omalley updated the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA fixed a merge problem REVISION DETAIL https://reviews.facebook.net/D2433 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Hide
        Phabricator added a comment -

        omalley updated the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA

        fixed one more mistake

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        Show
        Phabricator added a comment - omalley updated the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA fixed one more mistake REVISION DETAIL https://reviews.facebook.net/D2433 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Hide
        Owen O'Malley added a comment -

        I ran the unit tests and they passed.

        Show
        Owen O'Malley added a comment - I ran the unit tests and they passed.
        Hide
        Phabricator added a comment -

        ashutoshc has requested changes to the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".

        INLINE COMMENTS
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:545 Should this be named createMetaData()?
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1215 This is not returning string to string map, rather just Text.
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1219 Should this be named getMetadataValueOf() ?

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        BRANCH
        h-2898

        Show
        Phabricator added a comment - ashutoshc has requested changes to the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:545 Should this be named createMetaData()? ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1215 This is not returning string to string map, rather just Text. ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java:1219 Should this be named getMetadataValueOf() ? REVISION DETAIL https://reviews.facebook.net/D2433 BRANCH h-2898
        Hide
        Phabricator added a comment -

        omalley updated the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".
        Reviewers: JIRA, ashutoshc

        make ashutosh's suggested changes

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

        Show
        Phabricator added a comment - omalley updated the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". Reviewers: JIRA, ashutoshc make ashutosh's suggested changes REVISION DETAIL https://reviews.facebook.net/D2433 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Hide
        Phabricator added a comment -

        ashutoshc has accepted the revision "HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles".

        +1

        REVISION DETAIL
        https://reviews.facebook.net/D2433

        BRANCH
        h-2898

        Show
        Phabricator added a comment - ashutoshc has accepted the revision " HIVE-2898 [jira] Add nicer helper functions for adding and reading metadata from RCFiles". +1 REVISION DETAIL https://reviews.facebook.net/D2433 BRANCH h-2898
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Owen!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Owen!
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1338 (See https://builds.apache.org/job/Hive-trunk-h0.21/1338/)
        HIVE-2898: Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) (Revision 1306464)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306464
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1338 (See https://builds.apache.org/job/Hive-trunk-h0.21/1338/ ) HIVE-2898 : Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) (Revision 1306464) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306464 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2898: Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) (Revision 1306464)

        Result = ABORTED
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306464
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java
        • /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2898 : Add nicer helper functions for adding and reading metadata from RCFiles (Owen Omalley via Ashutosh Chauhan) (Revision 1306464) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306464 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development