Hive
  1. Hive
  2. HIVE-5104

HCatStorer fails to store boolean type

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: HCatalog
    • Labels:
      None
    • Release Note:
      Saving records that contain Boolean type's is now supported via Pig's HCatStorer.

      Description

      Unable to store boolean values to HCat table

      Assume in Hive you have two tables...
      CREATE TABLE btest(test as boolean);
      CREATE TABLE btest2(test as boolean);

      Then in Pig
      A = LOAD 'btest' USING org.apache.hcatalog.pig.HCatLoader();
      STORE A INTO 'btest2' USING org.apache.hcatalog.pig.HCatStorer();

      You will get an "ERROR 115: Unsupported type 5: in Pig's Schema"

      Checking HCatBaseStorer.java, the case for data types doesn't check for booleans. Might have been overlooked in adding boolean to Pig in 0.10

      1. HIVE-5104.2.patch
        8 kB
        Sushanth Sowmyan
      2. HIVE-5104.1.patch.txt
        7 kB
        Karl D. Gierach

        Activity

        Ron Frohock created issue -
        Hide
        Karl D. Gierach added a comment -

        code modified:

        1) hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        a) getHCatFSFromPigFS(...)
        b) getJavaObj(...)

        2) hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        a) testStoreFuncAllSimpleTypes()

        Show
        Karl D. Gierach added a comment - code modified: 1) hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java a) getHCatFSFromPigFS(...) b) getJavaObj(...) 2) hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java a) testStoreFuncAllSimpleTypes()
        Karl D. Gierach made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Reproduced In 0.11.0 [ 12323587 ]
        Hide
        Karl D. Gierach added a comment -

        the patch.

        based off github's "branch-0.11" branch.

        Show
        Karl D. Gierach added a comment - the patch. based off github's "branch-0.11" branch.
        Karl D. Gierach made changes -
        Attachment HIVE-5104.patch [ 12599242 ]
        Karl D. Gierach made changes -
        Release Note Saving records that contain Boolean type's is now supported via Pig's HCatStorer.
        Hide
        Karl D. Gierach added a comment -

        re-generated patch with correct git option "--no-prefix".

        Show
        Karl D. Gierach added a comment - re-generated patch with correct git option "--no-prefix".
        Karl D. Gierach made changes -
        Attachment HIVE-5104.1.patch.txt [ 12599335 ]
        Karl D. Gierach made changes -
        Attachment HIVE-5104.patch [ 12599242 ]
        Hide
        Sushanth Sowmyan added a comment -

        Hi,

        From looking through the patch, the patch looks good from a functionality perspective. Thank you for the patch.

        However, it does not seem to apply cleanly on trunk, and also has whitespace issues (trailing whitespaces) and needs regeneration.

        As part of HIVE-4869, all HCatalog jiras will be frozen for a couple of days as the package renaming effort happening there affects all jiras. Could you please regenerate your patch after that and re-apply?

        Show
        Sushanth Sowmyan added a comment - Hi, From looking through the patch, the patch looks good from a functionality perspective. Thank you for the patch. However, it does not seem to apply cleanly on trunk, and also has whitespace issues (trailing whitespaces) and needs regeneration. As part of HIVE-4869 , all HCatalog jiras will be frozen for a couple of days as the package renaming effort happening there affects all jiras. Could you please regenerate your patch after that and re-apply?
        Hide
        Sushanth Sowmyan added a comment -

        Actually, on manual application, the whitespace errors are easy enough to fix. I'm uploading a whitespace-corrected version of the patch and will try to get this patch in before the freeze if tests pass.

        Show
        Sushanth Sowmyan added a comment - Actually, on manual application, the whitespace errors are easy enough to fix. I'm uploading a whitespace-corrected version of the patch and will try to get this patch in before the freeze if tests pass.
        Hide
        Sushanth Sowmyan added a comment -

        One more required change was that "null" as presented in the modification to the test does not succeed, it needs to be NULL.

        Show
        Sushanth Sowmyan added a comment - One more required change was that "null" as presented in the modification to the test does not succeed, it needs to be NULL.
        Sushanth Sowmyan made changes -
        Attachment HIVE-5104.2.patch [ 12601229 ]
        Hide
        Sushanth Sowmyan added a comment -

        +1, committing.

        Show
        Sushanth Sowmyan added a comment - +1, committing.
        Hide
        Sushanth Sowmyan added a comment -

        Committed, thanks for the patch, Karl!

        Show
        Sushanth Sowmyan added a comment - Committed, thanks for the patch, Karl!
        Sushanth Sowmyan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Brock Noland made changes -
        Fix Version/s 0.12.0 [ 12324312 ]
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #403 (See https://builds.apache.org/job/Hive-trunk-hadoop2/403/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #403 (See https://builds.apache.org/job/Hive-trunk-hadoop2/403/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #83 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/83/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #83 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/83/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Ashutosh Chauhan made changes -
        Assignee Karl D. Gierach [ kgierach ]
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #150 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/150/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #150 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/150/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hive-trunk-h0.21 #2309 (See https://builds.apache.org/job/Hive-trunk-h0.21/2309/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hive-trunk-h0.21 #2309 (See https://builds.apache.org/job/Hive-trunk-h0.21/2309/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
        Ashutosh Chauhan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        5d 19h 41m 1 Karl D. Gierach 21/Aug/13 19:26
        Patch Available Patch Available Resolved Resolved
        13d 2h 5m 1 Sushanth Sowmyan 03/Sep/13 21:32
        Resolved Resolved Closed Closed
        42d 2h 56m 1 Ashutosh Chauhan 16/Oct/13 00:28

          People

          • Assignee:
            Karl D. Gierach
            Reporter:
            Ron Frohock
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development