Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16865 Handle replication bootstrap of large databases
  3. HIVE-16896

move replication load related work in semantic analysis phase to execution phase using a task

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      we want to not create too many tasks in memory in the analysis phase while loading data. Currently we load all the files in the bootstrap dump location as FileStatus[] and then iterate over it to load objects, we should rather move to

      org.apache.hadoop.fs.RemoteIterator<LocatedFileStatus>	listFiles(Path f, boolean recursive)
      

      which would internally batch and return values.

      additionally since we cant hand off partial tasks from analysis pahse => execution phase, we are going to move the whole repl load functionality to execution phase so we can better control creation/execution of tasks (not related to hive Task, we may get rid of ReplCopyTask)

      Additional consideration to take into account at the end of this jira is to see if we want to specifically do a multi threaded load of bootstrap dump.

      1. HIVE-16896.3.patch
        140 kB
        anishek
      2. HIVE-16896.2.patch
        120 kB
        anishek
      3. HIVE-16896.1.patch
        119 kB
        anishek

        Issue Links

          Activity

          Hide
          anishek anishek added a comment -

          We might have to have separate jira's to handle bootstrap repl load vs incremental repl load, with the wrapper or lazy top level task for both scenarios to work differently.

          Show
          anishek anishek added a comment - We might have to have separate jira's to handle bootstrap repl load vs incremental repl load, with the wrapper or lazy top level task for both scenarios to work differently.
          Hide
          anishek anishek added a comment -

          self reminder to add docs for "hive.repl.approx.max.load.tasks"

          Show
          anishek anishek added a comment - self reminder to add docs for "hive.repl.approx.max.load.tasks"
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12879776/HIVE-16896.1.patch

          SUCCESS: +1 due to 2 test(s) being added or modified.

          ERROR: -1 due to 8 failed/errored test(s), 11019 tests executed
          Failed tests:

          TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
          org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=158)
          org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168)
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin] (batchId=90)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6211/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6211/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6211/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 8 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12879776 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12879776/HIVE-16896.1.patch SUCCESS: +1 due to 2 test(s) being added or modified. ERROR: -1 due to 8 failed/errored test(s), 11019 tests executed Failed tests: TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=235) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=158) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin] (batchId=90) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6211/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6211/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6211/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed This message is automatically generated. ATTACHMENT ID: 12879776 - PreCommit-HIVE-Build
          Hide
          anishek anishek added a comment -

          not sure why the pull request is not shown here https://github.com/apache/hive/pull/214

          Show
          anishek anishek added a comment - not sure why the pull request is not shown here https://github.com/apache/hive/pull/214
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12880033/HIVE-16896.2.patch

          SUCCESS: +1 due to 2 test(s) being added or modified.

          ERROR: -1 due to 7 failed/errored test(s), 11041 tests executed
          Failed tests:

          TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
          org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168)
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin] (batchId=90)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6232/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6232/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6232/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 7 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12880033 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880033/HIVE-16896.2.patch SUCCESS: +1 due to 2 test(s) being added or modified. ERROR: -1 due to 7 failed/errored test(s), 11041 tests executed Failed tests: TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_load_requires_admin] (batchId=90) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6232/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6232/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6232/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed This message is automatically generated. ATTACHMENT ID: 12880033 - PreCommit-HIVE-Build
          Hide
          anishek anishek added a comment -

          reattaching the patch so build can be triggered.

          Show
          anishek anishek added a comment - reattaching the patch so build can be triggered.
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12880360/HIVE-16896.3.patch

          ERROR: -1 due to build exiting with an error

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6257/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6257/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6257/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests exited with: NonZeroExitCodeException
          Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
          2017-08-04 13:33:05.249
          + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
          + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
          + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
          + export 'MAVEN_OPTS=-Xmx1g '
          + MAVEN_OPTS='-Xmx1g '
          + cd /data/hiveptest/working/
          + tee /data/hiveptest/logs/PreCommit-HIVE-Build-6257/source-prep.txt
          + [[ false == \t\r\u\e ]]
          + mkdir -p maven ivy
          + [[ git = \s\v\n ]]
          + [[ git = \g\i\t ]]
          + [[ -z master ]]
          + [[ -d apache-github-source-source ]]
          + [[ ! -d apache-github-source-source/.git ]]
          + [[ ! -d apache-github-source-source ]]
          + date '+%Y-%m-%d %T.%3N'
          2017-08-04 13:33:05.251
          + cd apache-github-source-source
          + git fetch origin
          From https://github.com/apache/hive
             0bf8314..ceec583  master     -> origin/master
          + git reset --hard HEAD
          HEAD is now at 0bf8314 Add License Header for HIVE-17144
          + git clean -f -d
          Removing ql/src/test/queries/clientpositive/spark_union_merge.q
          Removing ql/src/test/results/clientpositive/spark/spark_union_merge.q.out
          + git checkout master
          Already on 'master'
          Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
            (use "git pull" to update your local branch)
          + git reset --hard origin/master
          HEAD is now at ceec583 HIVE-17208: Repl dump should pass in db/table information to authorization API (Daniel Dai, reviewed by Thejas Nair)
          + git merge --ff-only origin/master
          Already up-to-date.
          + date '+%Y-%m-%d %T.%3N'
          2017-08-04 13:33:11.444
          + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hiveptest/working/scratch/build.patch
          + [[ -f /data/hiveptest/working/scratch/build.patch ]]
          + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
          + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
          error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: No such file or directory
          error: a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java: No such file or directory
          error: a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WarehouseInstance.java: No such file or directory
          error: a/ql/if/queryplan.thrift: No such file or directory
          error: a/ql/src/gen/thrift/gen-cpp/queryplan_types.cpp: No such file or directory
          error: a/ql/src/gen/thrift/gen-cpp/queryplan_types.h: No such file or directory
          error: a/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/StageType.java: No such file or directory
          error: a/ql/src/gen/thrift/gen-php/Types.php: No such file or directory
          error: a/ql/src/gen/thrift/gen-py/queryplan/ttypes.py: No such file or directory
          error: a/ql/src/gen/thrift/gen-rb/queryplan_types.rb: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/Context.java: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java: No such file or directory
          error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java: No such file or directory
          error: a/ql/src/test/results/clientnegative/repl_load_requires_admin.q.out: No such file or directory
          The patch does not appear to apply with p0, p1, or p2
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12880360 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880360/HIVE-16896.3.patch ERROR: -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6257/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6257/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6257/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-08-04 13:33:05.249 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-6257/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-08-04 13:33:05.251 + cd apache-github-source-source + git fetch origin From https://github.com/apache/hive 0bf8314..ceec583 master -> origin/master + git reset --hard HEAD HEAD is now at 0bf8314 Add License Header for HIVE-17144 + git clean -f -d Removing ql/src/test/queries/clientpositive/spark_union_merge.q Removing ql/src/test/results/clientpositive/spark/spark_union_merge.q.out + git checkout master Already on 'master' Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded. (use "git pull" to update your local branch) + git reset --hard origin/master HEAD is now at ceec583 HIVE-17208: Repl dump should pass in db/table information to authorization API (Daniel Dai, reviewed by Thejas Nair) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-08-04 13:33:11.444 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: No such file or directory error: a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java: No such file or directory error: a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WarehouseInstance.java: No such file or directory error: a/ql/if/queryplan.thrift: No such file or directory error: a/ql/src/gen/thrift/gen-cpp/queryplan_types.cpp: No such file or directory error: a/ql/src/gen/thrift/gen-cpp/queryplan_types.h: No such file or directory error: a/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/StageType.java: No such file or directory error: a/ql/src/gen/thrift/gen-php/Types.php: No such file or directory error: a/ql/src/gen/thrift/gen-py/queryplan/ttypes.py: No such file or directory error: a/ql/src/gen/thrift/gen-rb/queryplan_types.rb: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/Context.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java: No such file or directory error: a/ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java: No such file or directory error: a/ql/src/test/results/clientnegative/repl_load_requires_admin.q.out: No such file or directory The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12880360 - PreCommit-HIVE-Build
          Hide
          anishek anishek added a comment -

          rebasing from master and resolving conflicts.

          Show
          anishek anishek added a comment - rebasing from master and resolving conflicts.
          Hide
          anishek anishek added a comment -

          Sankar Hariappan can you please do the review for this!

          Show
          anishek anishek added a comment - Sankar Hariappan can you please do the review for this!
          Hide
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12880608/HIVE-16896.3.patch

          SUCCESS: +1 due to 2 test(s) being added or modified.

          ERROR: -1 due to 7 failed/errored test(s), 10991 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] (batchId=56)
          org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168)
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99)
          org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179)
          org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6277/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6277/console
          Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6277/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 7 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12880608 - PreCommit-HIVE-Build

          Show
          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12880608/HIVE-16896.3.patch SUCCESS: +1 due to 2 test(s) being added or modified. ERROR: -1 due to 7 failed/errored test(s), 10991 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_uncompressed] (batchId=56) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning] (batchId=168) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] (batchId=99) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=179) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6277/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6277/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6277/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed This message is automatically generated. ATTACHMENT ID: 12880608 - PreCommit-HIVE-Build
          Hide
          sankarh Sankar Hariappan added a comment -

          +1.
          The 3.patch looks good to me.

          Show
          sankarh Sankar Hariappan added a comment - +1. The 3.patch looks good to me.
          Hide
          anishek anishek added a comment -

          Thanks Sankar Hariappan for the review. Thejas M Nair/Daniel Dai please commit the patch.

          Show
          anishek anishek added a comment - Thanks Sankar Hariappan for the review. Thejas M Nair / Daniel Dai please commit the patch.
          Hide
          daijy Daniel Dai added a comment -

          Patch pushed to master.

          Show
          daijy Daniel Dai added a comment - Patch pushed to master.
          Hide
          leftylev Lefty Leverenz added a comment -

          Doc note: This adds hive.repl.approx.max.load.tasks to HiveConf.java, so it needs to be documented in the wiki.

          Added a TODOC3.0 label.

          Show
          leftylev Lefty Leverenz added a comment - Doc note: This adds hive.repl.approx.max.load.tasks to HiveConf.java, so it needs to be documented in the wiki. Configuration Properties – Replication Added a TODOC3.0 label.
          Hide
          anishek anishek added a comment -

          Lefty Leverenz added the configuration doc.

          Show
          anishek anishek added a comment - Lefty Leverenz added the configuration doc.
          Hide
          leftylev Lefty Leverenz added a comment -

          Thanks for the doc, anishek. I'm suggesting some edits with HIVE-17426.

          Here's the direct link:

          Removed the TODOC3.0 label.

          Show
          leftylev Lefty Leverenz added a comment - Thanks for the doc, anishek . I'm suggesting some edits with HIVE-17426 . Here's the direct link: hive.repl.approx.max.load.tasks Removed the TODOC3.0 label.

            People

            • Assignee:
              anishek anishek
              Reporter:
              anishek anishek
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development