[IMPALA-10459] Remove workaround codes for MAPREDUCE-6441

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Epic Color: ghx-label-10
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 28/Jan/21 08:10
• Updated: 05/Feb/21 08:56
• Resolved: 05/Feb/21 08:56

Description

MAPREDUCE-6441 is resolved and is in our minicluster's version now. We can remove the workaround codes due to it:

$ git grep -n -C 2 'MAPREDUCE-6441'
bin/load-data.py-158-  # running MR jobs locally), we move the temporary directory into a unique
bin/load-data.py-159-  # directory via configuration. This block can be removed when
bin/load-data.py:160:  # https://issues.apache.org/jira/browse/MAPREDUCE-6441 is resolved.
bin/load-data.py-161-  hive_args = HIVE_ARGS
bin/load-data.py-162-  unique_dir = None
--
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java-190-  }
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java-191-
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java:192:  @Ignore("Ignored until MAPREDUCE-6441 is available in the toolchain")
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java-193-  @Test
fe/src/test/java/org/apache/impala/catalog/events/EventsProcessorStressTest.java-194-  public void testUsingRandomHiveQueries() throws Exception {
--
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-78-
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-79-  // we need to add small delay between the start of each query to work-around
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java:80:  // MAPREDUCE-6441
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-81-  private final Object delayLock_ = new Object();
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-82-  /**
--
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-1105-            LOG.info("Client {} running hive query set {}: {}", clientId, queryNumber,
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-1106-                query);
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java:1107:            // add a delay between the start of each query to work around MAPREDUCE-6441
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-1108-            synchronized (delayLock_) {
fe/src/test/java/org/apache/impala/util/RandomHiveQueryRunner.java-1109-              Thread.sleep(10);
--
tests/common/impala_test_suite.py-959-    # running MR jobs locally), we move the temporary directory into a unique
tests/common/impala_test_suite.py-960-    # directory via configuration. This workaround can be removed when
tests/common/impala_test_suite.py:961:    # https://issues.apache.org/jira/browse/MAPREDUCE-6441 is resolved.
tests/common/impala_test_suite.py-962-    # A similar workaround is used in bin/load-data.py.
tests/common/impala_test_suite.py-963-    tmpdir = None

Issue Links

• is required by: IMPALA-10458 load-data.sh fails to run Hive queries with IllegalArgumentException