Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: Metastore, Thrift API
    • Labels:
      None

      Description

      See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a better solution is probably to roll back the "fix" and change the API to use a primitive type as the map key (in a backwards-compatible manner).

      1. HIVE-4492.patch
        91 kB
        Ashutosh Chauhan
      2. HIVE-4492.1.patch.txt
        103 kB
        Samuel Yuan

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          Patch rebased on trunk.

          Show
          Ashutosh Chauhan added a comment - Patch rebased on trunk.
          Hide
          Ashutosh Chauhan added a comment -

          Samuel Yuan Can you take a look?

          Show
          Ashutosh Chauhan added a comment - Samuel Yuan Can you take a look?
          Hide
          Ashutosh Chauhan added a comment -
          Show
          Ashutosh Chauhan added a comment - Phabricator link: https://reviews.facebook.net/D12795
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12602038/HIVE-4492.patch

          ERROR: -1 due to 2 failed/errored test(s), 3086 tests executed
          Failed tests:

          org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable
          org.apache.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/660/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/660/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12602038/HIVE-4492.patch ERROR: -1 due to 2 failed/errored test(s), 3086 tests executed Failed tests: org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable org.apache.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/660/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/660/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Ashutosh Chauhan added a comment -

          Tests passed. May I request a committer to review the patch?

          Show
          Ashutosh Chauhan added a comment - Tests passed. May I request a committer to review the patch?
          Hide
          Carl Steinbach added a comment -

          +1

          Show
          Carl Steinbach added a comment - +1
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Samuel for initial patch. Thanks, Carl for review!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Samuel for initial patch. Thanks, Carl for review!
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2-ptest #89 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/89/)
          HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120)

          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #89 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/89/ ) HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120 ) /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop1-ptest #157 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/157/)
          HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120)

          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #157 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/157/ ) HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120 ) /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-h0.21 #2322 (See https://builds.apache.org/job/Hive-trunk-h0.21/2322/)
          HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120)

          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2322 (See https://builds.apache.org/job/Hive-trunk-h0.21/2322/ ) HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120 ) /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Hide
          Hudson added a comment -

          ABORTED: Integrated in Hive-trunk-hadoop2 #419 (See https://builds.apache.org/job/Hive-trunk-hadoop2/419/)
          HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120)

          • /hive/trunk/metastore/if/hive_metastore.thrift
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
          • /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
          • /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java
          • /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
          • /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
          • /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
          • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
          • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Show
          Hudson added a comment - ABORTED: Integrated in Hive-trunk-hadoop2 #419 (See https://builds.apache.org/job/Hive-trunk-hadoop2/419/ ) HIVE-4492 : Revert HIVE4322 (Samuel Yuan and Ashutosh Chauhan via Ashutosh Chauhan) (hashutosh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1521120 ) /hive/trunk/metastore/if/hive_metastore.thrift /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/listbucketingpruner/ListBucketingPruner.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
          Hide
          Thejas M Nair added a comment -

          This looks like an api backward compatibility fix that we should include in 0.12 . Ashutosh Chauhan Do you agree ?

          Show
          Thejas M Nair added a comment - This looks like an api backward compatibility fix that we should include in 0.12 . Ashutosh Chauhan Do you agree ?
          Hide
          Ashutosh Chauhan added a comment -

          Yup. Please apply to 0.12

          Show
          Ashutosh Chauhan added a comment - Yup. Please apply to 0.12
          Hide
          Thejas M Nair added a comment -

          Patch committed to 0.12 branch.

          Show
          Thejas M Nair added a comment - Patch committed to 0.12 branch.
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

            People

            • Assignee:
              Samuel Yuan
              Reporter:
              Samuel Yuan
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development