Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • None

    Description

      "slider.hdfs.keytab.dir": "...",
      "slider.am.login.keytab.name": "...",
      "slider.keytab.principal.name": "..."
      

      should be emitted into appConfig.json for Slider AM. Right now, they have to be added manually on a secure cluster.

      Attachments

        1. HIVE-12851.patch
          2 kB
          Sergey Shelukhin
        2. HIVE-12851.2.patch
          3 kB
          Gopal Vijayaraghavan

        Issue Links

        Activity

          sershe Sergey Shelukhin added a comment - Gunther Hagleitner Gopal Vijayaraghavan fyi

          The patch. I still need to test it, later today hopefully

          sershe Sergey Shelukhin added a comment - The patch. I still need to test it, later today hopefully

          -1 on the use of argparse - is a famously python3 module.

          The default centos6 python version is 2.7, which is what we expect at a normal install location today.

          gopalv Gopal Vijayaraghavan added a comment - -1 on the use of argparse - is a famously python3 module. The default centos6 python version is 2.7, which is what we expect at a normal install location today.

          argparse is supported for Python 2.3 and later... I think it's reasonable to expect users to install it if missing. It's also a standalone module, so it can be copied if missing. When I was testing it, it was present on Python 2.6.6 on RHEL 6.6 and CentOS 6.6 (2 separate clusters).

          sershe Sergey Shelukhin added a comment - argparse is supported for Python 2.3 and later... I think it's reasonable to expect users to install it if missing. It's also a standalone module, so it can be copied if missing. When I was testing it, it was present on Python 2.6.6 on RHEL 6.6 and CentOS 6.6 (2 separate clusters).

          QE automation uses it, which might be how you have it.

          gopalv Gopal Vijayaraghavan added a comment - QE automation uses it, which might be how you have it.

          Fix the --chaosmonkey option and add "s" suffix to the template formats.

          gopalv Gopal Vijayaraghavan added a comment - Fix the --chaosmonkey option and add "s" suffix to the template formats.
          hiveqa Hive QA added a comment -

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12782677/HIVE-12851.2.patch

          ERROR: -1 due to no test(s) being added or modified.

          ERROR: -1 due to 6 failed/errored test(s), 10019 tests executed
          Failed tests:

          TestHWISessionManager - did not produce a TEST-*.xml file
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
          org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
          org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
          org.apache.hive.jdbc.TestSSL.testSSLVersion
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6649/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6649/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6649/

          Messages:

          Executing org.apache.hive.ptest.execution.TestCheckPhase
          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 6 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12782677 - PreCommit-HIVE-TRUNK-Build

          hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12782677/HIVE-12851.2.patch ERROR: -1 due to no test(s) being added or modified. ERROR: -1 due to 6 failed/errored test(s), 10019 tests executed Failed tests: TestHWISessionManager - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse org.apache.hive.jdbc.TestSSL.testSSLVersion Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6649/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6649/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6649/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed This message is automatically generated. ATTACHMENT ID: 12782677 - PreCommit-HIVE-TRUNK-Build

          +1

          Sergey Shelukhin: you're right, python-argparse makes this much neater - I'll eat my objections

          I tested this on an insecure cluster as well, Slider ignores missing keytabs if there's no kerberos init.

          gopalv Gopal Vijayaraghavan added a comment - +1 Sergey Shelukhin : you're right, python-argparse makes this much neater - I'll eat my objections I tested this on an insecure cluster as well, Slider ignores missing keytabs if there's no kerberos init.

          Committed to master and branch-2.0 (since it's unlikely to affect any other code).

          sershe Sergey Shelukhin added a comment - Committed to master and branch-2.0 (since it's unlikely to affect any other code).
          asears Andrew Sears added a comment -

          With a Centos 6.5 minimal install and HDP 2.4 / Python 2.6.6 need to run the following to get argparse installed

          yum -y install epel-release
          pip install argparse

          asears Andrew Sears added a comment - With a Centos 6.5 minimal install and HDP 2.4 / Python 2.6.6 need to run the following to get argparse installed yum -y install epel-release pip install argparse

          Pip is probably not a repeatable install - using python-argparse RPMs is probably better

          Installed Packages
          Name        : python-argparse
          Arch        : noarch
          Version     : 1.2.1
          Release     : 2.el6
          Size        : 232 k
          Repo        : installed
          From repo   : epel
          Summary     : Optparse inspired command line parser for Python
          URL         : http://code.google.com/p/argparse/
          License     : Python
          
          gopalv Gopal Vijayaraghavan added a comment - Pip is probably not a repeatable install - using python-argparse RPMs is probably better Installed Packages Name : python-argparse Arch : noarch Version : 1.2.1 Release : 2.el6 Size : 232 k Repo : installed From repo : epel Summary : Optparse inspired command line parser for Python URL : http: //code.google.com/p/argparse/ License : Python
          asears Andrew Sears added a comment -

          Thanks Gopal, do you think this can be included as part of Hive packaging/LLAP setup or argparse dependency removed? Likely won't be an issue for customers running later versions of Python, though there are probably many who aren't and are in locked-down environments.

          Here's some information on the recommended way of installing argparse from CA.
          https://docops.ca.com/ca-advanced-authentication/8-1/EN/upgrading/verify-prerequisties/install-argparse-without-pip

          I think wiki needs some documentation around LLAP setup & dependencies.

          asears Andrew Sears added a comment - Thanks Gopal, do you think this can be included as part of Hive packaging/LLAP setup or argparse dependency removed? Likely won't be an issue for customers running later versions of Python, though there are probably many who aren't and are in locked-down environments. Here's some information on the recommended way of installing argparse from CA. https://docops.ca.com/ca-advanced-authentication/8-1/EN/upgrading/verify-prerequisties/install-argparse-without-pip I think wiki needs some documentation around LLAP setup & dependencies.

          People

            sershe Sergey Shelukhin Assign to me
            sershe Sergey Shelukhin
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack