Hive
  1. Hive
  2. HIVE-6679

HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.13.0, 0.14.0
    • Fix Version/s: 1.1.0
    • Component/s: HiveServer2
    • Labels:

      Description

      HiveServer2 should support configurable the server side socket read timeout and TCP keep-alive option. Metastore server already support this (and the so is the old hive server).
      We now have multiple client connectivity options like Kerberos, Delegation Token (Digest-MD5), Plain SASL, Plain SASL with SSL and raw sockets. The configuration should be applicable to all types (if possible).

      1. HIVE-6679.6.patch
        6 kB
        Vaibhav Gumashta
      2. HIVE-6679.5.patch
        8 kB
        Vaibhav Gumashta
      3. HIVE-6679.4.patch
        8 kB
        Vaibhav Gumashta
      4. HIVE-6679.3.patch
        8 kB
        Vaibhav Gumashta
      5. HIVE-6679.2.patch.txt
        10 kB
        Navis
      6. HIVE-6679.1.patch.txt
        10 kB
        Navis

        Issue Links

          Activity

          Hide
          Lefty Leverenz added a comment -

          Patch 1 adds configuration parameters hive.server2.read.socket.timeout and hive.server2.tcp.keepalive to HiveConf.java and hive-default.xml.template.

          (This comment is a search aid and a reminder to document them in the wiki after the patch is committed.)

          Show
          Lefty Leverenz added a comment - Patch 1 adds configuration parameters hive.server2.read.socket.timeout and hive.server2.tcp.keepalive to HiveConf.java and hive-default.xml.template. (This comment is a search aid and a reminder to document them in the wiki after the patch is committed.)
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12652339/HIVE-6679.1.patch.txt

          ERROR: -1 due to 1 failed/errored test(s), 5669 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/587/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/587/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-587/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12652339

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12652339/HIVE-6679.1.patch.txt ERROR: -1 due to 1 failed/errored test(s), 5669 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/587/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/587/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-587/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12652339
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12666680/HIVE-6679.2.patch.txt

          SUCCESS: +1 6171 tests passed

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/658/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/658/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-658/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12666680

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12666680/HIVE-6679.2.patch.txt SUCCESS: +1 6171 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/658/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/658/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-658/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12666680
          Hide
          Vaibhav Gumashta added a comment -

          Patch will need some rebasing.

          Show
          Vaibhav Gumashta added a comment - Patch will need some rebasing.
          Hide
          Vaibhav Gumashta added a comment -

          Patch rebased on trunk. Thejas M Nair This will be a good addition to 14.1. What do you think?

          Show
          Vaibhav Gumashta added a comment - Patch rebased on trunk. Thejas M Nair This will be a good addition to 14.1. What do you think?
          Hide
          Thejas M Nair added a comment -

          The 3.patch does not have the http transport changes that were there in 2.patch. Is that intentional ?
          I think enabling this by default makes sense, instead of having users set this if they run into issues. A timeout of around couple of minutes might be reasonable default.

          Show
          Thejas M Nair added a comment - The 3.patch does not have the http transport changes that were there in 2.patch. Is that intentional ? I think enabling this by default makes sense, instead of having users set this if they run into issues. A timeout of around couple of minutes might be reasonable default.
          Hide
          Vaibhav Gumashta added a comment -

          Thejas M Nair Yes, that was intentional since we already have a max idle time config for http mode. Let me enable the keepalive config by default.
          Regarding the timeout, do you think couple of minutes should be fine? I think we should keep the default pretty high since we don't want the server to throw SocketTimeoutException prematurely (note that the timeout doesn't close the socket, it just throws an exception in hitting the timeout on blocking calls).

          Show
          Vaibhav Gumashta added a comment - Thejas M Nair Yes, that was intentional since we already have a max idle time config for http mode. Let me enable the keepalive config by default. Regarding the timeout, do you think couple of minutes should be fine? I think we should keep the default pretty high since we don't want the server to throw SocketTimeoutException prematurely (note that the timeout doesn't close the socket, it just throws an exception in hitting the timeout on blocking calls).
          Hide
          Vaibhav Gumashta added a comment -

          Patch .4 sets keepAlive to true by default for TCP sockets.

          Show
          Vaibhav Gumashta added a comment - Patch .4 sets keepAlive to true by default for TCP sockets.
          Hide
          Thejas M Nair added a comment -

          +1

          Show
          Thejas M Nair added a comment - +1
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12687288/HIVE-6679.3.patch

          ERROR: -1 due to 1 failed/errored test(s), 6703 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2080/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2080/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2080/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12687288 - PreCommit-HIVE-TRUNK-Build

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12687288/HIVE-6679.3.patch ERROR: -1 due to 1 failed/errored test(s), 6703 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2080/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2080/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2080/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12687288 - PreCommit-HIVE-TRUNK-Build
          Hide
          Prasad Mujumdar added a comment -

          Vaibhav Gumashta Apache Thrift has added support for keep alive (THRIFT-2788). The trunk is already upgraded to use Thrift 0.9.2 which has that patch.

          Show
          Prasad Mujumdar added a comment - Vaibhav Gumashta Apache Thrift has added support for keep alive ( THRIFT-2788 ). The trunk is already upgraded to use Thrift 0.9.2 which has that patch.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12687301/HIVE-6679.4.patch

          ERROR: -1 due to 1 failed/errored test(s), 6703 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_covar_pop
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2081/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2081/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2081/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12687301 - PreCommit-HIVE-TRUNK-Build

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12687301/HIVE-6679.4.patch ERROR: -1 due to 1 failed/errored test(s), 6703 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_covar_pop Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2081/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2081/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2081/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12687301 - PreCommit-HIVE-TRUNK-Build
          Hide
          Vaibhav Gumashta added a comment -

          Test failure not related.

          Show
          Vaibhav Gumashta added a comment - Test failure not related.
          Hide
          Vaibhav Gumashta added a comment -

          Thanks for pointing out Prasad Mujumdar. I'll limit the scope of this one to 14.1 in that case.

          Show
          Vaibhav Gumashta added a comment - Thanks for pointing out Prasad Mujumdar . I'll limit the scope of this one to 14.1 in that case.
          Hide
          Lefty Leverenz added a comment -

          Doc note: Patch 4 adds two configuration parameters (which are different from the two in patch 1): hive.server2.tcp.socket.blocking.timeout and hive.server2.tcp.socket.keepalive.

          Code review: The time specification for hive.server2.tcp.socket.blocking.timeout could be done with a TimeValidator like others in HiveConf.java since 0.14.0, for example:

              METASTORE_CLIENT_CONNECT_RETRY_DELAY("hive.metastore.client.connect.retry.delay", "1s",
                  new TimeValidator(TimeUnit.SECONDS),
          
          Show
          Lefty Leverenz added a comment - Doc note: Patch 4 adds two configuration parameters (which are different from the two in patch 1): hive.server2.tcp.socket.blocking.timeout and hive.server2.tcp.socket.keepalive . Code review: The time specification for hive.server2.tcp.socket.blocking.timeout could be done with a TimeValidator like others in HiveConf.java since 0.14.0, for example: METASTORE_CLIENT_CONNECT_RETRY_DELAY( "hive.metastore.client.connect.retry.delay" , "1s" , new TimeValidator(TimeUnit.SECONDS),
          Hide
          Vaibhav Gumashta added a comment -

          Lefty Leverenz Patch 5 addresses review comments.

          Show
          Vaibhav Gumashta added a comment - Lefty Leverenz Patch 5 addresses review comments.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12687857/HIVE-6679.5.patch

          ERROR: -1 due to 5 failed/errored test(s), 6713 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby3_map
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby3_map_multi_distinct
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
          org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2117/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2117/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2117/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 5 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12687857 - PreCommit-HIVE-TRUNK-Build

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12687857/HIVE-6679.5.patch ERROR: -1 due to 5 failed/errored test(s), 6713 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby3_map org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby3_map_multi_distinct org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2117/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2117/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2117/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed This message is automatically generated. ATTACHMENT ID: 12687857 - PreCommit-HIVE-TRUNK-Build
          Hide
          Thejas M Nair added a comment -

          The socket timeout part of this still relevant for trunk. I think we should commit that part in trunk.

          Show
          Thejas M Nair added a comment - The socket timeout part of this still relevant for trunk. I think we should commit that part in trunk.
          Hide
          Lefty Leverenz added a comment -

          HiveConf.java parameter definitions look good.

          +1 (for configuration parameters only)

          Show
          Lefty Leverenz added a comment - HiveConf.java parameter definitions look good. +1 (for configuration parameters only)
          Hide
          Vaibhav Gumashta added a comment -

          Committed to branch 14. Thejas M Nair I'll create a new patch for trunk with just the socket timeout code.

          Show
          Vaibhav Gumashta added a comment - Committed to branch 14. Thejas M Nair I'll create a new patch for trunk with just the socket timeout code.
          Hide
          Lefty Leverenz added a comment -

          I'll create a new patch for trunk with just the socket timeout code.

          Does this mean hive.server2.tcp.socket.keepalive will only exist in 0.14.1, not in 0.15?

          Show
          Lefty Leverenz added a comment - I'll create a new patch for trunk with just the socket timeout code. Does this mean hive.server2.tcp.socket.keepalive will only exist in 0.14.1, not in 0.15?
          Hide
          Vaibhav Gumashta added a comment -

          Lefty Leverenz Yes, as of now. I'll add a patch for 15 shortly.

          Show
          Vaibhav Gumashta added a comment - Lefty Leverenz Yes, as of now. I'll add a patch for 15 shortly.
          Hide
          Vaibhav Gumashta added a comment -

          Thejas M Nair V6 for 1.1.0 & trunk

          Show
          Vaibhav Gumashta added a comment - Thejas M Nair V6 for 1.1.0 & trunk
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12696010/HIVE-6679.6.patch

          SUCCESS: +1 7421 tests passed

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2623/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2623/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2623/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12696010 - PreCommit-HIVE-TRUNK-Build

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12696010/HIVE-6679.6.patch SUCCESS: +1 7421 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2623/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2623/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2623/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12696010 - PreCommit-HIVE-TRUNK-Build
          Hide
          Lefty Leverenz added a comment -

          Currently this shows status Patch Available and resolution Unresolved because there's a patch for 1.1.0, although the 1.0.0 patch was committed and it is included in the release. (I checked HiveConf.java for hive.server2.tcp.socket.blocking.timeout and hive.server2.tcp.socket.keepalive but didn't check other patch changes.)

          Unfortunately the 1.0.0 release notes linked to the downloads page don't include this issue. There may be other JIRA issues that were fixed in 1.0.0 but fell off the back of the cart when 0.15 was changed to 1.1 and then to 1.1.0.

          Show
          Lefty Leverenz added a comment - Currently this shows status Patch Available and resolution Unresolved because there's a patch for 1.1.0, although the 1.0.0 patch was committed and it is included in the release. (I checked HiveConf.java for hive.server2.tcp.socket.blocking.timeout and hive.server2.tcp.socket.keepalive but didn't check other patch changes.) Unfortunately the 1.0.0 release notes linked to the downloads page don't include this issue. There may be other JIRA issues that were fixed in 1.0.0 but fell off the back of the cart when 0.15 was changed to 1.1 and then to 1.1.0. 1.0.0 release notes

            People

            • Assignee:
              Navis
              Reporter:
              Prasad Mujumdar
            • Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development