Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Transactions
    • None

    Description

      HIVE-17361 only allows 00000_0 and 0000_0_copy_1. Should it support arbitrary names?
      If so, should it sort them and rename 0000_0, 0001_0, etc?
      This is probably a lot easier than changing the whole code base to assign proper 'bucket' (writerId) everywhere Acid reads such file.

      Attachments

        1. HIVE-18125.01.patch
          6 kB
          Eugene Koifman
        2. HIVE-18125.02.patch
          8 kB
          Eugene Koifman

        Issue Links

          Activity

            hiveqa Hive QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
                  Prechecks
            0 findbugs 0m 0s Findbugs executables are not available.
            +1 @author 0m 1s The patch does not contain any @author tags.
                  master Compile Tests
            +1 mvninstall 7m 9s master passed
            +1 compile 1m 1s master passed
            +1 checkstyle 0m 37s master passed
            +1 javadoc 0m 54s master passed
                  Patch Compile Tests
            +1 mvninstall 1m 23s the patch passed
            +1 compile 1m 0s the patch passed
            +1 javac 1m 0s the patch passed
            -1 checkstyle 0m 38s ql: The patch generated 3 new + 354 unchanged - 0 fixed = 357 total (was 354)
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 javadoc 1m 1s the patch passed
                  Other Tests
            +1 asflicense 0m 12s The patch does not generate ASF License warnings.
            14m 13s



            Subsystem Report/Notes
            Optional Tests asflicense javac javadoc findbugs checkstyle compile
            uname Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux
            Build tool maven
            Personality /data/hiveptest/working/yetus/dev-support/hive-personality.sh
            git revision master / b7be4ac
            Default Java 1.8.0_111
            checkstyle http://104.198.109.242/logs//PreCommit-HIVE-Build-8220/yetus/diff-checkstyle-ql.txt
            modules C: ql U: ql
            Console output http://104.198.109.242/logs//PreCommit-HIVE-Build-8220/yetus.txt
            Powered by Apache Yetus http://yetus.apache.org

            This message was automatically generated.

            hiveqa Hive QA added a comment - -1 overall Vote Subsystem Runtime Comment       Prechecks 0 findbugs 0m 0s Findbugs executables are not available. +1 @author 0m 1s The patch does not contain any @author tags.       master Compile Tests +1 mvninstall 7m 9s master passed +1 compile 1m 1s master passed +1 checkstyle 0m 37s master passed +1 javadoc 0m 54s master passed       Patch Compile Tests +1 mvninstall 1m 23s the patch passed +1 compile 1m 0s the patch passed +1 javac 1m 0s the patch passed -1 checkstyle 0m 38s ql: The patch generated 3 new + 354 unchanged - 0 fixed = 357 total (was 354) +1 whitespace 0m 0s The patch has no whitespace issues. +1 javadoc 1m 1s the patch passed       Other Tests +1 asflicense 0m 12s The patch does not generate ASF License warnings. 14m 13s Subsystem Report/Notes Optional Tests asflicense javac javadoc findbugs checkstyle compile uname Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux Build tool maven Personality /data/hiveptest/working/yetus/dev-support/hive-personality.sh git revision master / b7be4ac Default Java 1.8.0_111 checkstyle http://104.198.109.242/logs//PreCommit-HIVE-Build-8220/yetus/diff-checkstyle-ql.txt modules C: ql U: ql Console output http://104.198.109.242/logs//PreCommit-HIVE-Build-8220/yetus.txt Powered by Apache Yetus http://yetus.apache.org This message was automatically generated.
            hiveqa Hive QA added a comment -

            Here are the results of testing the latest attachment:
            https://issues.apache.org/jira/secure/attachment/12901814/HIVE-18125.01.patch

            SUCCESS: +1 due to 1 test(s) being added or modified.

            ERROR: -1 due to 19 failed/errored test(s), 11530 tests executed
            Failed tests:

            org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48)
            org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
            org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt11] (batchId=71)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2] (batchId=152)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=169)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=160)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[quotedid_smb] (batchId=157)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160)
            org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat] (batchId=178)
            org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part] (batchId=93)
            org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_data_into_acid] (batchId=92)
            org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_10] (batchId=138)
            org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketsortoptimize_insert_7] (batchId=128)
            org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=120)
            org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113)
            org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=226)
            org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead (batchId=233)
            

            Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8220/testReport
            Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8220/console
            Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8220/

            Messages:

            Executing org.apache.hive.ptest.execution.TestCheckPhase
            Executing org.apache.hive.ptest.execution.PrepPhase
            Executing org.apache.hive.ptest.execution.YetusPhase
            Executing org.apache.hive.ptest.execution.ExecutionPhase
            Executing org.apache.hive.ptest.execution.ReportingPhase
            Tests exited with: TestsFailedException: 19 tests failed
            

            This message is automatically generated.

            ATTACHMENT ID: 12901814 - PreCommit-HIVE-Build

            hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12901814/HIVE-18125.01.patch SUCCESS: +1 due to 1 test(s) being added or modified. ERROR: -1 due to 19 failed/errored test(s), 11530 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt11] (batchId=71) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=169) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=160) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[quotedid_smb] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat] (batchId=178) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part] (batchId=93) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_data_into_acid] (batchId=92) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_10] (batchId=138) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketsortoptimize_insert_7] (batchId=128) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] (batchId=120) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=226) org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead (batchId=233) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8220/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8220/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8220/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 19 tests failed This message is automatically generated. ATTACHMENT ID: 12901814 - PreCommit-HIVE-Build
            ekoifman Eugene Koifman added a comment -

            fixed via HIVE-18516

            ekoifman Eugene Koifman added a comment - fixed via  HIVE-18516

            This jira is resolved and released with Hive 3.0 If you find an issue with it, please create a new jira.

            ashutoshc Ashutosh Chauhan added a comment - This jira is resolved and released with Hive 3.0 If you find an issue with it, please create a new jira.

            People

              ekoifman Eugene Koifman
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: