Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9483

Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: documentation
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      If WebHDFS is secured with SSL, then you can use "swebhdfs" as the scheme in a URL to access it. The current documentation does not state this anywhere.

      1. HDFS-9483.001.patch
        0.8 kB
        Surendra Singh Lilhore
      2. HDFS-9483.002.patch
        1 kB
        Surendra Singh Lilhore
      3. HDFS-9483.patch
        0.8 kB
        Surendra Singh Lilhore

        Issue Links

          Activity

          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Attached patch. Please review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Attached patch. Please review..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 mvnsite 1m 23s hadoop-hdfs in trunk failed.
          -1 mvnsite 1m 0s hadoop-hdfs in the patch failed.
          -1 whitespace 0m 0s The patch has 58 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 asflicense 0m 43s Patch does not generate ASF License warnings.
          3m 30s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775832/HDFS-9483.patch
          JIRA Issue HDFS-9483
          Optional Tests asflicense mvnsite
          uname Linux eb3408016b7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cbc7b6b
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/whitespace-eol.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 31MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13774/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 mvnsite 1m 23s hadoop-hdfs in trunk failed. -1 mvnsite 1m 0s hadoop-hdfs in the patch failed. -1 whitespace 0m 0s The patch has 58 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 asflicense 0m 43s Patch does not generate ASF License warnings. 3m 30s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775832/HDFS-9483.patch JIRA Issue HDFS-9483 Optional Tests asflicense mvnsite uname Linux eb3408016b7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cbc7b6b mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13774/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 31MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13774/console This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          mvnsite failure is unrelated to this patch. It is because of compilation failure in TestEncryptionZones.java

          [ERROR] /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java:[1444,24] error: method getCurrentTrashDir in class FsShell cannot be applied to given types;
          
          Show
          surendrasingh Surendra Singh Lilhore added a comment - mvnsite failure is unrelated to this patch. It is because of compilation failure in TestEncryptionZones.java [ERROR] /testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java:[1444,24] error: method getCurrentTrashDir in class FsShell cannot be applied to given types;
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Resubmitting patch to trigger jenkins.

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Resubmitting patch to trigger jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvnsite 1m 26s trunk passed
          +1 mvnsite 1m 8s the patch passed
          -1 whitespace 0m 0s The patch has 58 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 asflicense 0m 37s Patch does not generate ASF License warnings.
          3m 27s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch
          JIRA Issue HDFS-9483
          Optional Tests asflicense mvnsite
          uname Linux 4fc5691467d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 50edcb9
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13816/artifact/patchprocess/whitespace-eol.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 32MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13816/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvnsite 1m 26s trunk passed +1 mvnsite 1m 8s the patch passed -1 whitespace 0m 0s The patch has 58 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 asflicense 0m 37s Patch does not generate ASF License warnings. 3m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch JIRA Issue HDFS-9483 Optional Tests asflicense mvnsite uname Linux 4fc5691467d1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 50edcb9 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13816/artifact/patchprocess/whitespace-eol.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 32MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13816/console This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Chris Nauroth, Please can you review...

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Chris Nauroth , Please can you review...
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 7m 37s trunk passed
          +1 mvnsite 0m 59s trunk passed
          +1 mvnsite 0m 54s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          10m 20s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch
          JIRA Issue HDFS-9483
          Optional Tests asflicense mvnsite
          uname Linux 8c9669470813 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e7ed05e
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14887/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 7m 37s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 10m 20s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch JIRA Issue HDFS-9483 Optional Tests asflicense mvnsite uname Linux 8c9669470813 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e7ed05e modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14887/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Thanks Surendra Singh Lilhore for working on this.. and thanks to Chris Nauroth for reporting this..

          I am thinking, we should update in distcp documenation also. what do you think..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - Thanks Surendra Singh Lilhore for working on this.. and thanks to Chris Nauroth for reporting this.. I am thinking, we should update in distcp documenation also. what do you think..?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 6m 37s trunk passed
          +1 mvnsite 0m 50s trunk passed
          +1 mvnsite 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          8m 57s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-9483
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch
          Optional Tests asflicense mvnsite
          uname Linux 67d526dcdd4e 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6eb700e
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16856/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 37s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvnsite 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 8m 57s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9483 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776616/HDFS-9483.001.patch Optional Tests asflicense mvnsite uname Linux 67d526dcdd4e 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16856/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Brahma Reddy Battula for review and comment..

          I am thinking, we should update in distcp documenation also. what do you think..?

          yes, we should update in distcp documenation also.

          I will update patch..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Brahma Reddy Battula for review and comment.. I am thinking, we should update in distcp documenation also. what do you think..? yes, we should update in distcp documenation also. I will update patch..
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Attached updated patch..
          Please review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Attached updated patch.. Please review..
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          Surendra Singh Lilhore thanks for updating the patch. changes looks fine to me..Triggered the jenkins..

          Chris Nauroth does it needs some more update.like SSL config link.?

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited Surendra Singh Lilhore thanks for updating the patch. changes looks fine to me..Triggered the jenkins.. Chris Nauroth does it needs some more update.like SSL config link.?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          0 mvndep 1m 38s Maven dependency ordering for branch
          +1 mvninstall 8m 38s trunk passed
          +1 mvnsite 1m 28s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvnsite 1m 9s the patch passed
          +1 whitespace 0m 1s The patch has no whitespace issues.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          14m 7s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-9483
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840560/HDFS-9483.002.patch
          Optional Tests asflicense mvnsite
          uname Linux fe53cdd5a1f0 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5d5614f
          modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17675/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. 0 mvndep 1m 38s Maven dependency ordering for branch +1 mvninstall 8m 38s trunk passed +1 mvnsite 1m 28s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvnsite 1m 9s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 14m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-9483 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840560/HDFS-9483.002.patch Optional Tests asflicense mvnsite uname Linux fe53cdd5a1f0 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5d5614f modules C: hadoop-hdfs-project/hadoop-hdfs hadoop-tools/hadoop-distcp U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17675/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          pinging Chris Nauroth

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited pinging Chris Nauroth
          Hide
          cnauroth Chris Nauroth added a comment -

          I'm +1 for patch 002. Brahma Reddy Battula, you had suggested perhaps a link to SSL configuration information. Unfortunately, the only place I'm aware of with that information is the Encrypted Shuffle page, and it would be kind of odd to link to that from DistCp. I'll hold off committing in case you have further comments.

          Show
          cnauroth Chris Nauroth added a comment - I'm +1 for patch 002. Brahma Reddy Battula , you had suggested perhaps a link to SSL configuration information. Unfortunately, the only place I'm aware of with that information is the Encrypted Shuffle page, and it would be kind of odd to link to that from DistCp. I'll hold off committing in case you have further comments.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Chris Nauroth thanks your reply..Agreed with you.Me too +1 on latest patch..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Chris Nauroth thanks your reply..Agreed with you.Me too +1 on latest patch..
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk, branch-2 and branch-2.8. Surendra Singh Lilhore, thank you for contributing the patch. Brahma Reddy Battula, thank you for help with the code review.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk, branch-2 and branch-2.8. Surendra Singh Lilhore , thank you for contributing the patch. Brahma Reddy Battula , thank you for help with the code review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11077 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11077/)
          HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme (cnauroth: rev 4c8f9e130230457fc897ed7a2a09e14d078be90a)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
          • (edit) hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11077 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11077/ ) HDFS-9483 . Documentation does not cover use of "swebhdfs" as URL scheme (cnauroth: rev 4c8f9e130230457fc897ed7a2a09e14d078be90a) (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md (edit) hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Chris Nauroth for review and commit. Thanks Brahma Reddy Battula for review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Chris Nauroth for review and commit. Thanks Brahma Reddy Battula for review..

            People

            • Assignee:
              surendrasingh Surendra Singh Lilhore
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development