Hi Robert Justice - we may want to consider the use of the CredentialProvider API that will be committed soon.
See HADOOP-10607. This isn't mutually exclusive with the password file approach as there are plans to fallback to existing password files in certain components. However, the abstraction of the API is best realized through the new Configuration.getPassword(String name) method. This will allow you to ask for a configuration item that you know is a password and it will check for an aliased credential based on the name through the CredentialProvider API. If the name is not resolved into a credential from a provider then it falls back to the config file.

The extra hop of the separate file isn't a problem but it isn't encapsulated by the getPassword method going into Configuration.

Just something to keep in mind.

Great idea Larry McCay. I'm open for whatever solution is the most secure for users. Some users will require the password file to be encrypted or in a 3rd party store, based on security policies and this would be a solution to that.

These should be seen as complementary issues.

I've added a patch that provides this ability akin to Sqoop's mechanism (minus the encrypted/obfuscated file loader options, as those could be better handled by Larry's proposal).

This would be useful in the immediate future, until what Larry proposes can be compatibly added to Hive in future upon completion upstream.

Please review.

Overall: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12648646/HIVE-7175.patch

ERROR: -1 due to 7 failed/errored test(s), 5511 tests executed
Failed tests:

org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/399/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/399/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-399/

Messages:

Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed

This message is automatically generated.

ATTACHMENT ID: 12648646

Hi Hive QA, none of the failed tests appear related to the small additive change specific to BeeLine done here. These tests appear to be generally failing on trunk, and are not caused by this patch. Let me know if I am wrong.

[~dr.wendell.urth] Thanks for working on this. Could you please provide a review board entry for the patch?

Also could you add a nice message given an exception in obtainPasswordFromFile.

I just realized that this is the users' LDAP password.
It would be unfortunate to have to leave this laying around in various places unless absolutely necessary.

Does the beeline CLI currently allow for using the java Console to collect the password from the user?

I understand that for scripting type purposes we may need another collection mechanism but for usecases with a user and console available the users' passwords should not be persisted outside of the directory itself when it can be avoided.

For cases where it can not be avoided the side file approach is certainly better than on the command line itself in terms of visibility.

Dr. Wendell Urth Thanks for the work so far. Do you plan to incorporate the follow-up feedback? Let me know if you don't have free cycles - I can pitch in.

Patch based on trunk.

cc Thejas M Nair

+1
Thanks Dr. Wendell Urth and Vaibhav Gumashta!

Patch for 0.13.0 in case anyone needs it.

Reuploading for precommit run.

Overall: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12696265/HIVE-7175.1.patch

ERROR: -1 due to 2 failed/errored test(s), 7423 tests executed
Failed tests:

org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hive.beeline.TestBeelineArgParsing.testPasswordFileArgs

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2644/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2644/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2644/

Messages:

Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed

This message is automatically generated.

ATTACHMENT ID: 12696265 - PreCommit-HIVE-TRUNK-Build

Overall: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12696544/HIVE-7175.2.patch

SUCCESS: +1 7476 tests passed

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2666/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2666/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2666/

Messages:

Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase

This message is automatically generated.

ATTACHMENT ID: 12696544 - PreCommit-HIVE-TRUNK-Build

Thanks for helping on this improvement to beeline: Dr. Wendell Urth, Vaibhav Gumashta, and Larry McCay

Patch committed to trunk. Thanks Dr. Wendell Urth for the patch, thanks Larry McCay, Robert Justice and Thejas M Nair for reviewing.

Doc note: This should be documented in the Beeline section of HiveServer2 Clients, with version information for 1.2.0 and a link to this issue for the 0.13 patch.

• HiveServer2 Clients – Beeline Command Options

This issue has been fixed and released as part of the 1.2.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

Documented in the wiki page mentioned by Lefty Leverenz - https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30758725&selectedPageVersions=138&selectedPageVersions=136