Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FileContext uses Hdfs instead of DistributedFileSystem. We need to add delegation token APIs in Hdfs class as well.

      1. HDFS-1442.2.patch
        9 kB
        Jitendra Nath Pandey
      2. HDFS-1442.4.patch
        8 kB
        Jitendra Nath Pandey
      3. HDFS-1442.5.patch
        9 kB
        Jitendra Nath Pandey
      4. HDFS-1442.6.patch
        9 kB
        Jitendra Nath Pandey
      5. HDFS-1442.7.patch
        10 kB
        Jitendra Nath Pandey
      6. HDFS-1442.8.patch
        4 kB
        Jitendra Nath Pandey
      7. HDFS-1442.9.patch
        9 kB
        Jitendra Nath Pandey

        Issue Links

          Activity

          Jitendra Nath Pandey created issue -
          Jitendra Nath Pandey made changes -
          Field Original Value New Value
          Link This issue is blocked by HADOOP-6994 [ HADOOP-6994 ]
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.2.patch [ 12456641 ]
          Hide
          Jitendra Nath Pandey added a comment -

          This patch is based against latest trunk and adds a new test.

          Show
          Jitendra Nath Pandey added a comment - This patch is based against latest trunk and adds a new test.
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.4.patch [ 12475266 ]
          Hide
          Jitendra Nath Pandey added a comment -

          New patch corresponding to updates on HADOOP-6994 patch.

          Show
          Jitendra Nath Pandey added a comment - New patch corresponding to updates on HADOOP-6994 patch.
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.5.patch [ 12475755 ]
          Hide
          Jitendra Nath Pandey added a comment -

          New patch corresponds to latest patch for HADOOP-6994.

          Show
          Jitendra Nath Pandey added a comment - New patch corresponds to latest patch for HADOOP-6994 .
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.6.patch [ 12476156 ]
          Hide
          Suresh Srinivas added a comment -

          Comments:

          1. Please add some description to TestFileContextResolveAfs
          2. Unrelated to your patch, there is warning for Hdfs.getNext() in throws clause. Could you fix it by importing java.util.NoSuchElementException and not use full path in the getNext() method implementation.

          +1 for the patch.

          Show
          Suresh Srinivas added a comment - Comments: Please add some description to TestFileContextResolveAfs Unrelated to your patch, there is warning for Hdfs.getNext() in throws clause. Could you fix it by importing java.util.NoSuchElementException and not use full path in the getNext() method implementation. +1 for the patch.
          Hide
          Jitendra Nath Pandey added a comment -

          Updated patch addressing the comments.

          Show
          Jitendra Nath Pandey added a comment - Updated patch addressing the comments.
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.7.patch [ 12476164 ]
          Hide
          Jitendra Nath Pandey added a comment -

          Updated patch doesn't change the renew and cancel API in DistributedFileSystem.

          Show
          Jitendra Nath Pandey added a comment - Updated patch doesn't change the renew and cancel API in DistributedFileSystem.
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.8.patch [ 12476184 ]
          Hide
          Jitendra Nath Pandey added a comment -

          Forgot to add the test file in previous patch.

          Show
          Jitendra Nath Pandey added a comment - Forgot to add the test file in previous patch.
          Jitendra Nath Pandey made changes -
          Attachment HDFS-1442.9.patch [ 12476192 ]
          Hide
          Suresh Srinivas added a comment -

          +1 for the patch.

          Show
          Suresh Srinivas added a comment - +1 for the patch.
          Hide
          Jitendra Nath Pandey added a comment -

          test patch results
          [exec] -1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 3 new or modified tests.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
          [exec]
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          [exec]
          [exec] -1 system test framework. The patch failed system test framework compile.

          Show
          Jitendra Nath Pandey added a comment - test patch results [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] -1 system test framework. The patch failed system test framework compile.
          Hide
          Jitendra Nath Pandey added a comment -

          ant test was run manually. All tests passed except TestHDFSTrash, TestLargeBlock and TestReplaceDatanodeOnFailure which are unrelated to this patch.

          Show
          Jitendra Nath Pandey added a comment - ant test was run manually. All tests passed except TestHDFSTrash, TestLargeBlock and TestReplaceDatanodeOnFailure which are unrelated to this patch.
          Hide
          Jitendra Nath Pandey added a comment -

          System test framework failed with errors
          [exec] [iajc] /Users/jitendra/workspace/commit/hadoop-hdfs/src/java/org/apache/hadoop/fs/Hdfs.java:396 [error] The method getDelegationTokens(String) of type Hdfs must override or implement a supertype method
          [exec] [iajc] public List<Token<?>> getDelegationTokens(String renewer) throws IOException {
          [exec] [iajc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          [exec] [iajc] /Users/jitendra/workspace/commit/hadoop-hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:817 [error] The method getDelegationTokens(String) of type DistributedFileSystem must override or implement a supertype method
          [exec] [iajc] public List<Token<?>> getDelegationTokens(String renewer) throws IOException {

          The super-class methods are implemented in HADOOP-6994 patch which will be committed together with this patch. It seems super-class methods were not resolved or detected from internal repository. However, fault injection tests passed when I ran ant test with resolvers=internal.

          Show
          Jitendra Nath Pandey added a comment - System test framework failed with errors [exec] [iajc] /Users/jitendra/workspace/commit/hadoop-hdfs/src/java/org/apache/hadoop/fs/Hdfs.java:396 [error] The method getDelegationTokens(String) of type Hdfs must override or implement a supertype method [exec] [iajc] public List<Token<?>> getDelegationTokens(String renewer) throws IOException { [exec] [iajc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [exec] [iajc] /Users/jitendra/workspace/commit/hadoop-hdfs/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:817 [error] The method getDelegationTokens(String) of type DistributedFileSystem must override or implement a supertype method [exec] [iajc] public List<Token<?>> getDelegationTokens(String renewer) throws IOException { The super-class methods are implemented in HADOOP-6994 patch which will be committed together with this patch. It seems super-class methods were not resolved or detected from internal repository. However, fault injection tests passed when I ran ant test with resolvers=internal.
          Hide
          Jitendra Nath Pandey added a comment -

          I have committed this.

          Show
          Jitendra Nath Pandey added a comment - I have committed this.
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.23.0 [ 12315571 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #590 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/590/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #590 (See https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/590/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )

            People

            • Assignee:
              Jitendra Nath Pandey
              Reporter:
              Jitendra Nath Pandey
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development