[ARROW-9719] [Doc][Python] Better document the new pa.fs.HadoopFileSystem

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 6.0.0
• Component/s: Documentation, Python
• Labels: filesystem pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25774
• Assignee: Antoine Pitrou
• Reporter: Joris Van den Bossche
• Created: 13/Aug/20 08:55
• Updated: 11/Jan/23 08:08
• Resolved: 04/Aug/21 15:02

Description

In order to deprecate the "legacy" pa.hdfs.HadoopFileSystem and pa.hdfs.connect(), we should ensure that the documentation for the new filesystem interface is fully up to date:

• The docstring of pyarrow.fs.HadoopFileSystem misses some keywords (that are included in the docstring of pa.hdfs.HaddopFileSystem)
• The user guide about the legacy one (https://arrow.apache.org/docs/python/filesystems_deprecated.html) contains more content, which should be added to the new user guide page as well (if still correct for the new filesystem)

Issue Links

• links to: GitHub Pull Request #10867