Details
-
Type:
Sub-task
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.13.0
-
Fix Version/s: 0.13.0
-
Component/s: HiveServer2
-
Labels:None
-
Release Note:
Description
HIVE-4617 provides support for async execution in HS2. The client gets an operation handle which it can poll to check on the operation status. However, the polling frequency is entirely left to the client which can be resource inefficient. Long polling will solve this, by blocking the client request to check the operation status for a configurable amount of time (a new HS2 config) if the data is not available, but responding immediately if the data is available.
Issue Links
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Phab link: https://reviews.facebook.net/D12801
+1 for latest patch in phabricator. Can you please upload that patch to jira as well ?
Sorry missed out attaching the patch. Thanks for the review!
[~vaibhavgumashta] Version 5 of the patch won't apply cleanly on trunk. Can you please rebase the patch and post a review request on RB?
One concern I have with the patch is that the delay period affects everyone who calls SQLOperation.getStatus(). Ideally we would be able to limit this behavior to remote clients only. I'm worried that at some point in the future we're going to want to have a housekeeping or monitoring thread that periodically calls getStatus() on all active operations, and we don't want getStatus() to block in these types of situations. Would it be practical to relocate this logic to the CLIService layer so that it only impacts CLIService clients?
Carl Steinbach Sure, let me rebase on the trunk. You can look at the patch here: https://reviews.facebook.net/D12801, but I can create a request on RB if you prefer that.
Actually the delay period only affects the async SQLOperation (and returns immediately if the async call has the results) and doesn't change the blocking calls. Also, as far as I understand the use case you are pointing to is for a cleanup type of operation, where we might want to clean up the entire session based on whether the client is active or not. In that case, the cleanup thread might end up checking the last active time of the session rather than the operation state and based on whether it exceeds some threshold, delete the session from the server.
Moving the long polling logic to CLIService might involve changing the Operation API, which I was trying to avoid until we get to a specific use case. But if you feel otherwise, I can definitely make the change.
Thanks for looking at the patch!
Patch rebased on trunk
Overall: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12612471/HIVE-5217.D12801.6.patch
SUCCESS: +1 4577 tests passed
Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/168/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/168/console
Messages:
Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase
This message is automatically generated.
ATTACHMENT ID: 12612471
Carl Steinbach What Vaibhav is saying sounds reasonable to me. It wouldn't be too hard to add support for a non long polling getOperationStatus call in future, specially a non rpc call. We can make the change when we have such a use case.
It wouldn't be too hard to add support for a non long polling getOperationStatus call in future, specially a non rpc call.
Adding a getOperationStatusInternal() or getOperationStatusNonLongPoll() method to work around this problem is exactly the sort of thing I want to avoid. This patch implements a service layer feature, so I think it makes sense that the implementation belongs in the service layer as well. In CLIService.getOperationStatus() you have access to the SessionManager, and from that you can easily get the Session object, the Operation object, the operation type, and the Session's configuration. As an added bonus you also get the OperationHandle which makes it easier to emit useful log messages that reference the session and operation IDs.
I posted the patch to reviewboard and left some comments: https://reviews.apache.org/r/15337/
Based on the test case included with the patch it looks like hive.server2.long.polling.timeout sets a lower bound on the total round trip time required to execute a getOperationStatus RPC, e.g. if polling.timeout = 5000 my query can finish after a second but getOperationStatus will still block for another four seconds before returning. Is this accurate?
Thanks Carl Steinbach. I'll look the the feedback here and on rb.
Meanwhile on your last comment: getOperationStatus returns after hive.server2.long.polling.timeout expires or the query completes - whichever happens first. So if polling.timeout = 5000 and the query finishes in 1000, the call to getOperationStatus will return in 1000. However, in the test case, the test query we are running is very short - therefore to test the behavior of the long polling timeout, we're explicitly setting the config to a smaller time than default, which is 5000; otherwise if the query takes 3000, it will return state FINISHED and we won't be able to test the timeout.
Couldn't update the other rb link as I wasn't the creator. Link for latest changes: https://reviews.apache.org/r/15435/
[~vaibhavgumashta] Is the patch ready for review?
Carl Steinbach Yes, you can take a look now. Thanks.
I left some more comments on reviewboard.
Are you planning to move the long polling timeout logic from SQLOperation.getState() to CLIService?
Carl Steinbach Uploaded a patch based on the feedback.
Regarding moving the long polling timeout logic, I have some comments where I'll appreciate your feedback:
1. Moving that logic to CLIService would mean accessing some of the operation logic within CLIService, and from the general design of CLIService it looks like we wanted to avoid that.
2. This will also mean that the async nature moves to Operation (runAsync, backgroundHandle) from SQLOperation.
What are your thoughts? Thanks for the feedback and discussion so far!
Carl Steinbach Look forward to your feedback on this one. Thanks!
Carl Steinbach Thejas M Nair Any further thoughts on the discussion so far? Thanks!
Carl Steinbach Thejas M Nair I just uploaded a patch which moves the polling timeout logic to CLIService. It would be awesome if you can take a look. Thanks in advance!
Overall: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12629519/HIVE-5217.5.patch
Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1383/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1383/console
Messages:
**** This message was trimmed, see log for full details ****
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-shims-0.20S ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/0.20S/target/hive-shims-0.20S-0.13.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/shims/hive-shims-0.20S/0.13.0-SNAPSHOT/hive-shims-0.20S-0.13.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/0.20S/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/shims/hive-shims-0.20S/0.13.0-SNAPSHOT/hive-shims-0.20S-0.13.0-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Shims 0.23 0.13.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-shims-0.23 ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23 (includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ hive-shims-0.23 ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/src/main/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-shims-0.23 ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-shims-0.23 ---
[INFO] Compiling 3 source files to /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/classes
[WARNING] Note: /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java uses or overrides a deprecated API.
[WARNING] Note: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ hive-shims-0.23 ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/src/test/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-shims-0.23 ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/tmp
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/warehouse
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/tmp/conf
[copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-shims-0.23 ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-shims-0.23 ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-shims-0.23 ---
[INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/hive-shims-0.23-0.13.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-shims-0.23 ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/target/hive-shims-0.23-0.13.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/shims/hive-shims-0.23/0.13.0-SNAPSHOT/hive-shims-0.23-0.13.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/0.23/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/shims/hive-shims-0.23/0.13.0-SNAPSHOT/hive-shims-0.23-0.13.0-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Shims 0.13.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-shims ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator (includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ hive-shims ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/src/main/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-shims ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-shims ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ hive-shims ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/src/test/resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-shims ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/tmp
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/warehouse
[mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/tmp/conf
[copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-shims ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-shims ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-shims ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/hive-shims-0.13.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-shims ---
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/target/hive-shims-0.13.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-shims/0.13.0-SNAPSHOT/hive-shims-0.13.0-SNAPSHOT.jar
[INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/shims/aggregator/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-shims/0.13.0-SNAPSHOT/hive-shims-0.13.0-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Common 0.13.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-common ---
[INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/common (includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (generate-version-annotation) @ hive-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-source) @ hive-common ---
[INFO] Source directory: /data/hive-ptest/working/apache-svn-trunk-source/common/src/gen added.
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ hive-common ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-common ---
[INFO] Compiling 38 source files to /data/hive-ptest/working/apache-svn-trunk-source/common/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,5] invalid method declaration; return type required
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,39] illegal start of type
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,76] illegal start of type
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,82] ';' expected
[INFO] 4 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Hive .............................................. SUCCESS [4.656s]
[INFO] Hive Ant Utilities ................................ SUCCESS [10.872s]
[INFO] Hive Shims Common ................................. SUCCESS [4.671s]
[INFO] Hive Shims 0.20 ................................... SUCCESS [2.808s]
[INFO] Hive Shims Secure Common .......................... SUCCESS [3.859s]
[INFO] Hive Shims 0.20S .................................. SUCCESS [1.806s]
[INFO] Hive Shims 0.23 ................................... SUCCESS [5.922s]
[INFO] Hive Shims ........................................ SUCCESS [0.531s]
[INFO] Hive Common ....................................... FAILURE [7.598s]
[INFO] Hive Serde ........................................ SKIPPED
[INFO] Hive Metastore .................................... SKIPPED
[INFO] Hive Query Language ............................... SKIPPED
[INFO] Hive Service ...................................... SKIPPED
[INFO] Hive JDBC ......................................... SKIPPED
[INFO] Hive Beeline ...................................... SKIPPED
[INFO] Hive CLI .......................................... SKIPPED
[INFO] Hive Contrib ...................................... SKIPPED
[INFO] Hive HBase Handler ................................ SKIPPED
[INFO] Hive HCatalog ..................................... SKIPPED
[INFO] Hive HCatalog Core ................................ SKIPPED
[INFO] Hive HCatalog Pig Adapter ......................... SKIPPED
[INFO] Hive HCatalog Server Extensions ................... SKIPPED
[INFO] Hive HCatalog Webhcat Java Client ................. SKIPPED
[INFO] Hive HCatalog Webhcat ............................. SKIPPED
[INFO] Hive HCatalog HBase Storage Handler ............... SKIPPED
[INFO] Hive HWI .......................................... SKIPPED
[INFO] Hive ODBC ......................................... SKIPPED
[INFO] Hive Shims Aggregator ............................. SKIPPED
[INFO] Hive TestUtils .................................... SKIPPED
[INFO] Hive Packaging .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 45.887s
[INFO] Finished at: Tue Feb 18 10:48:11 EST 2014
[INFO] Final Memory: 29M/71M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-common: Compilation failure: Compilation failure:
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,5] invalid method declaration; return type required
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,39] illegal start of type
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,76] illegal start of type
[ERROR] /data/hive-ptest/working/apache-svn-trunk-source/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:[1857,82] ';' expected
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hive-common
+ exit 1
'
This message is automatically generated.
ATTACHMENT ID: 12629519
Incorporating Thejas' feedback given on RB.
+1
Overall: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12630053/HIVE-5217.6.patch
ERROR: -1 due to 2 failed/errored test(s), 5173 tests executed
Failed tests:
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hcatalog.hbase.TestHBaseDirectOutputFormat.org.apache.hcatalog.hbase.TestHBaseDirectOutputFormat
Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1439/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1439/console
Messages:
Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed
This message is automatically generated.
ATTACHMENT ID: 12630053
Patch committed to trunk.
Thanks for the contribution Vaibhav!
Can you add see if the release note looks OK ? Please feel free to edit it.
hive.server2.long.polling.timeout is documented in the wiki here:
Here is the idea: http://tools.ietf.org/html/draft-loreto-http-bidirectional-07.