Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11750

Python HDFS: add Kerberos authentication support

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-py-hadoop

    Description

      The HDFS client used by Beam supports Kerberos.

      Initial idea: add a flag --hdfs_client that defaults to "INSECURE" and also accepts "KERBEROS". This flag will control initialization of self._hdfs_client.

      HDFS client docs:
      https://hdfscli.readthedocs.io/en/latest/api.html#module-hdfs.ext.kerberos

      The HDFS client seems to use this Kerberos library:
      https://pypi.org/project/requests-kerberos/

      Attachments

        Activity

          People

            Unassigned Unassigned
            udim Udi Meiri
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: