[ARROW-3503] [Python] Allow config hadoop_bin in pyarrow hdfs.py

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Later
• Component/s: Python
• Labels: filesystem pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19822
• Assignee: Unassigned
• Reporter: Wenbo Zhao
• Created: 12/Oct/18 13:44
• Updated: 11/Jan/23 07:27
• Resolved: 04/Aug/21 09:05

Description

Currently, the hadoop_bin is either from `HADOOP_HOME` or the `hadoop` command. 

https://github.com/apache/arrow/blob/master/python/pyarrow/hdfs.py#L130

However, in some of environment setup, hadoop_bin could be some other location. Can we do something like 

 

if 'HADOOP_BIN' in os.environ:
    hadoop_bin = os.environ['HADOOP_BIN']
elif 'HADOOP_HOME' in os.environ:
    hadoop_bin = '{0}/bin/hadoop'.format(os.environ['HADOOP_HOME'])
else:
    hadoop_bin = 'hadoop'

 

 

Issue Links

• links to: GitHub Pull Request #2748