Solr
  1. Solr
  2. SOLR-6372

HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.4, 6.0
    • Fix Version/s: 4.10, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      When setting the UserGroupInformation's Configuration, HdfsDirectorFactory creates a new Configuration, see here:
      https://github.com/apache/lucene-solr/blob/44f0dd7599901ba4d3625b5da8b998858e198d67/solr/core/src/java/org/apache/solr/core/HdfsDirectoryFactory.java#L306

      The issue with this is when it's used to set the UserGroupInformation configuration, it can overwrite any existing configuration, which may have other (server) settings like kerberos rules. Ideally there would be separate client/servers interfaces so this wouldn't be an issue. But lacking that, using the passed in Configuration allows the user to always specify all necessary settings.

      1. SOLR-6372.patch
        2 kB
        Gregory Chanan

        Activity

        Hide
        Gregory Chanan added a comment -

        Here's a patch that uses the supplied Configuration. Unfortunately, we don't have any tests currently that use kerberos-based HDFS, so I tested this manually on a cluster I have. I'll file a JIRA for adding those tests, I think it should be possible using Hadoop's MiniKDC.

        Show
        Gregory Chanan added a comment - Here's a patch that uses the supplied Configuration. Unfortunately, we don't have any tests currently that use kerberos-based HDFS, so I tested this manually on a cluster I have. I'll file a JIRA for adding those tests, I think it should be possible using Hadoop's MiniKDC.
        Hide
        Gregory Chanan added a comment - - edited

        Filed SOLR-6373 for adding unit tests.

        Show
        Gregory Chanan added a comment - - edited Filed SOLR-6373 for adding unit tests.
        Hide
        ASF subversion and git services added a comment -

        Commit 1617852 from Mark Miller in branch 'dev/trunk'
        [ https://svn.apache.org/r1617852 ]

        SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.

        Show
        ASF subversion and git services added a comment - Commit 1617852 from Mark Miller in branch 'dev/trunk' [ https://svn.apache.org/r1617852 ] SOLR-6372 : HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.
        Hide
        ASF subversion and git services added a comment -

        Commit 1617853 from Mark Miller in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1617853 ]

        SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.

        Show
        ASF subversion and git services added a comment - Commit 1617853 from Mark Miller in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1617853 ] SOLR-6372 : HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos.
        Hide
        Mark Miller added a comment -

        Thanks Greg!

        Show
        Mark Miller added a comment - Thanks Greg!

          People

          • Assignee:
            Mark Miller
            Reporter:
            Gregory Chanan
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development