Hive
  1. Hive
  2. HIVE-5104

HCatStorer fails to store boolean type

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: HCatalog
    • Labels:
      None
    • Release Note:
      Saving records that contain Boolean type's is now supported via Pig's HCatStorer.

      Description

      Unable to store boolean values to HCat table

      Assume in Hive you have two tables...
      CREATE TABLE btest(test as boolean);
      CREATE TABLE btest2(test as boolean);

      Then in Pig
      A = LOAD 'btest' USING org.apache.hcatalog.pig.HCatLoader();
      STORE A INTO 'btest2' USING org.apache.hcatalog.pig.HCatStorer();

      You will get an "ERROR 115: Unsupported type 5: in Pig's Schema"

      Checking HCatBaseStorer.java, the case for data types doesn't check for booleans. Might have been overlooked in adding boolean to Pig in 0.10

      1. HIVE-5104.2.patch
        8 kB
        Sushanth Sowmyan
      2. HIVE-5104.1.patch.txt
        7 kB
        Karl D. Gierach

        Activity

        Hide
        Karl D. Gierach added a comment -

        code modified:

        1) hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        a) getHCatFSFromPigFS(...)
        b) getJavaObj(...)

        2) hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        a) testStoreFuncAllSimpleTypes()

        Show
        Karl D. Gierach added a comment - code modified: 1) hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java a) getHCatFSFromPigFS(...) b) getJavaObj(...) 2) hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java a) testStoreFuncAllSimpleTypes()
        Hide
        Karl D. Gierach added a comment -

        the patch.

        based off github's "branch-0.11" branch.

        Show
        Karl D. Gierach added a comment - the patch. based off github's "branch-0.11" branch.
        Hide
        Karl D. Gierach added a comment -

        re-generated patch with correct git option "--no-prefix".

        Show
        Karl D. Gierach added a comment - re-generated patch with correct git option "--no-prefix".
        Hide
        Sushanth Sowmyan added a comment -

        Hi,

        From looking through the patch, the patch looks good from a functionality perspective. Thank you for the patch.

        However, it does not seem to apply cleanly on trunk, and also has whitespace issues (trailing whitespaces) and needs regeneration.

        As part of HIVE-4869, all HCatalog jiras will be frozen for a couple of days as the package renaming effort happening there affects all jiras. Could you please regenerate your patch after that and re-apply?

        Show
        Sushanth Sowmyan added a comment - Hi, From looking through the patch, the patch looks good from a functionality perspective. Thank you for the patch. However, it does not seem to apply cleanly on trunk, and also has whitespace issues (trailing whitespaces) and needs regeneration. As part of HIVE-4869 , all HCatalog jiras will be frozen for a couple of days as the package renaming effort happening there affects all jiras. Could you please regenerate your patch after that and re-apply?
        Hide
        Sushanth Sowmyan added a comment -

        Actually, on manual application, the whitespace errors are easy enough to fix. I'm uploading a whitespace-corrected version of the patch and will try to get this patch in before the freeze if tests pass.

        Show
        Sushanth Sowmyan added a comment - Actually, on manual application, the whitespace errors are easy enough to fix. I'm uploading a whitespace-corrected version of the patch and will try to get this patch in before the freeze if tests pass.
        Hide
        Sushanth Sowmyan added a comment -

        One more required change was that "null" as presented in the modification to the test does not succeed, it needs to be NULL.

        Show
        Sushanth Sowmyan added a comment - One more required change was that "null" as presented in the modification to the test does not succeed, it needs to be NULL.
        Hide
        Sushanth Sowmyan added a comment -

        +1, committing.

        Show
        Sushanth Sowmyan added a comment - +1, committing.
        Hide
        Sushanth Sowmyan added a comment -

        Committed, thanks for the patch, Karl!

        Show
        Sushanth Sowmyan added a comment - Committed, thanks for the patch, Karl!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #403 (See https://builds.apache.org/job/Hive-trunk-hadoop2/403/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #403 (See https://builds.apache.org/job/Hive-trunk-hadoop2/403/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #83 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/83/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #83 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/83/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #150 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/150/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #150 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/150/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hive-trunk-h0.21 #2309 (See https://builds.apache.org/job/Hive-trunk-h0.21/2309/)
        HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839)

        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java
        • /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hive-trunk-h0.21 #2309 (See https://builds.apache.org/job/Hive-trunk-h0.21/2309/ ) HIVE-5104 : HCatStorer fails to store boolean type (Karl D. Gierach via Sushanth Sowmyan) (khorgath: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1519839 ) /hive/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatBaseStorer.java /hive/trunk/hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorer.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          People

          • Assignee:
            Karl D. Gierach
            Reporter:
            Ron Frohock
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development