Hadoop Common
  1. Hadoop Common
  2. HADOOP-6545

Cached FileSystem objects can lead to wrong token being used in setting up connections

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: security
    • Labels:
      None

      Description

      The FileSystem class caches the filesystem objects that it creates for users. For some cases, e.g., if the filesystem object is actually a DistributedFileSystem, it also has an associated RPC client and hence an UGI for the respective user. This could lead to issues to do with using the right credentials when connecting with the namenode. The credentials in the UGI is never updated (even if the user in question now has new credentials) and in case the cached UGI's credentials have expired, this would lead to authentication error whenever there is a re-authentication (in the process of re-establishing connection to the namenode).

      1. 6545-1.patch
        2 kB
        Devaraj Das
      2. 6545-2.patch
        5 kB
        Devaraj Das
      3. 6545-bp20.patch
        5 kB
        Devaraj Das

        Issue Links

          Activity

          Hide
          Devaraj Das added a comment -

          Looks like the correct solution is to have the UGI as part of the key in the FileSystem cache. Attaching a patch that has this change.

          Show
          Devaraj Das added a comment - Looks like the correct solution is to have the UGI as part of the key in the FileSystem cache. Attaching a patch that has this change.
          Hide
          Devaraj Das added a comment -

          Oh I changed the toString in FileSystem.Cache.Key to have UGI.toString within braces. This is because UGI's toString could be "effective-user via real-user", and i wanted to associate the two together via the braces..

          Show
          Devaraj Das added a comment - Oh I changed the toString in FileSystem.Cache.Key to have UGI.toString within braces. This is because UGI's toString could be "effective-user via real-user", and i wanted to associate the two together via the braces..
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12435830/6545-1.patch
          against trunk revision 909806.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435830/6545-1.patch against trunk revision 909806. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/5/console This message is automatically generated.
          Hide
          Devaraj Das added a comment -

          This patch adds tests.

          Show
          Devaraj Das added a comment - This patch adds tests.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12435891/6545-2.patch
          against trunk revision 910169.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435891/6545-2.patch against trunk revision 910169. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/361/console This message is automatically generated.
          Hide
          Owen O'Malley added a comment -

          +1

          Show
          Owen O'Malley added a comment - +1
          Hide
          Devaraj Das added a comment -

          I just committed this.

          Show
          Devaraj Das added a comment - I just committed this.
          Hide
          Devaraj Das added a comment -

          The backport for Y20. Not for commit here.

          Show
          Devaraj Das added a comment - The backport for Y20. Not for commit here.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #179 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/179/)
          . Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #179 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/179/ ) . Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #256 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/256/)
          . Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #256 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/256/ ) . Changes the Key for the FileSystem cache to be UGI. Contributed by Devaraj Das.

            People

            • Assignee:
              Devaraj Das
              Reporter:
              Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development