[ARROW-350] Add Kerberos support to HDFS shim

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: C++, Python
• External issue URL: https://github.com/apache/arrow/issues/15852
• Assignee: Christopher Aycock
• Reporter: Christopher Aycock
• Created: 27/Oct/16 14:50
• Updated: 11/Jan/23 07:08
• Resolved: 29/Oct/16 01:14

Description

Speaking with wesmckinn, it would be really helpful to have Kerberos support in our HDFS logic. This should be straightforward; I would just need to switch to hdfsBuilderConnect() in the shim.

On a side note, is there a reason we aren't using Pivotal's libhdfs3? It uses RPCs natively rather than JNI.

https://github.com/Pivotal-Data-Attic/pivotalrd-libhdfs3

Dask has Python wrappers for this.

https://github.com/dask/hdfs3