Hive
  1. Hive
  2. HIVE-6749

Turn hive.auto.convert.join.use.nonstaged off by default

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0
    • Fix Version/s: 0.13.0
    • Component/s: Configuration
    • Labels:
      None
    1. HIVE-6749.patch
      1 kB
      Ashutosh Chauhan
    2. HIVE-6749.1.patch
      885 kB
      Ashutosh Chauhan

      Issue Links

        Activity

        Hide
        Lefty Leverenz added a comment -

        The changed default value for hive.auto.convert.join.use.nonstaged is documented in the wiki here:

        I added a comment to HIVE-6586 so the correct default value of hive.auto.convert.join.use.nonstaged won't get lost in the shuffle when HIVE-6037 changes HiveConf.java.

        Show
        Lefty Leverenz added a comment - The changed default value for hive.auto.convert.join.use.nonstaged is documented in the wiki here: Configuration Properties – hive.auto.convert.join.use.nonstaged I added a comment to HIVE-6586 so the correct default value of hive.auto.convert.join.use.nonstaged won't get lost in the shuffle when HIVE-6037 changes HiveConf.java.
        Hide
        Ashutosh Chauhan added a comment -

        Committed to 0.13 & branch. Thanks, Navis for review.

        Show
        Ashutosh Chauhan added a comment - Committed to 0.13 & branch. Thanks, Navis for review.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12638270/HIVE-6749.1.patch

        ERROR: -1 due to 2 failed/errored test(s), 5542 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2
        org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2092/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2092/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 2 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12638270

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12638270/HIVE-6749.1.patch ERROR: -1 due to 2 failed/errored test(s), 5542 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_2 org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2092/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2092/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12638270
        Hide
        Navis added a comment -

        Distributing small aliases(about 500M max) through DistributedCache makes busy works between DNs, and it was not good for the whole system. But +1 for the idea. It's not matured, yet.

        Show
        Navis added a comment - Distributing small aliases(about 500M max) through DistributedCache makes busy works between DNs, and it was not good for the whole system. But +1 for the idea. It's not matured, yet.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12636822/HIVE-6749.patch

        ERROR: -1 due to 109 failed/errored test(s), 5488 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join28
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join29
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_rearrange
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_convert_join
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join25
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join26
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join27
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join28
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join29
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join30
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join31
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32_lessSize
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join33
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join34
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join35
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join36
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join37
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_map_ppr
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_star
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_distinct
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_hook
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_memcheck
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_join_union
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_exclude_join
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_transform_hint
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin_noskew
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_15
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats11
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subq_where_serialization
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_having
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_bucket_mapjoin_mismatch1
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_sortmerge_mapjoin_mismatch_1
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1968/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1968/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 109 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12636822

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12636822/HIVE-6749.patch ERROR: -1 due to 109 failed/errored test(s), 5488 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join28 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join29 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_explain_rearrange org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_convert_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join27 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join28 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join29 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join30 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join31 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32_lessSize org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join33 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join34 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join35 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join36 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join37 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_map_ppr org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_reorder4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_star org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_hook org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_memcheck org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_subquery2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multiMapJoin2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_multi_join_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_reduce_deduplicate_exclude_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_transform_hint org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin_noskew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subq_where_serialization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_bucket_mapjoin_mismatch1 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_sortmerge_mapjoin_mismatch_1 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1968/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1968/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 109 tests failed This message is automatically generated. ATTACHMENT ID: 12636822
        Hide
        Ashutosh Chauhan added a comment -

        Attaching patch. Navis Let me know what you think?

        Show
        Ashutosh Chauhan added a comment - Attaching patch. Navis Let me know what you think?
        Hide
        Ashutosh Chauhan added a comment -

        It should be turned off by default because :

        • Its not clear it will always be more performant than local hashtable generation. When this kicks in, there will be three DataNode (which has replicas of smaller table) serving 100s of map tasks which are doing remote reads. Compare to this, DistributedCache distributes small file as well as localize, which will be local read. There might be some case that nonstaged is faster, but I dont think that will be a general case.
        • Seems like this will not work with security. Because we are reading smaller table directly now from hdfs, whose tokens are not obtained.
        • Its a new feature, so lets give it some time to mature.
        Show
        Ashutosh Chauhan added a comment - It should be turned off by default because : Its not clear it will always be more performant than local hashtable generation. When this kicks in, there will be three DataNode (which has replicas of smaller table) serving 100s of map tasks which are doing remote reads. Compare to this, DistributedCache distributes small file as well as localize, which will be local read. There might be some case that nonstaged is faster, but I dont think that will be a general case. Seems like this will not work with security. Because we are reading smaller table directly now from hdfs, whose tokens are not obtained. Its a new feature, so lets give it some time to mature.

          People

          • Assignee:
            Ashutosh Chauhan
            Reporter:
            Ashutosh Chauhan
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development