Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4553

Webhdfs will NPE on some unexpected response codes

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.7
    • Component/s: webhdfs
    • Labels:
      None

      Description

      Webhdfs will NPE in validateResponse if an expected response code is received w/o a json payload containing the exception.

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #545 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/545/)
          HDFS-4553. Webhdfs will NPE on some unexpected response codes. Contributed by Daryn Sharp. (Revision 1452994)

          Result = FAILURE
          kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1452994
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #545 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/545/ ) HDFS-4553 . Webhdfs will NPE on some unexpected response codes. Contributed by Daryn Sharp. (Revision 1452994) Result = FAILURE kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1452994 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
          Hide
          Daryn Sharp added a comment -

          Yes, it is! I thought I had filed a similar jira but couldn't find it.

          Show
          Daryn Sharp added a comment - Yes, it is! I thought I had filed a similar jira but couldn't find it.
          Hide
          Suresh Srinivas added a comment -

          Is HDFS-4490 fixed by this change? If so can you link it to this jira and mark that as closed.

          Show
          Suresh Srinivas added a comment - Is HDFS-4490 fixed by this change? If so can you link it to this jira and mark that as closed.
          Hide
          Kihwal Lee added a comment -

          I've committed this to branch-0.23. Thanks for the work, Daryn.

          Show
          Kihwal Lee added a comment - I've committed this to branch-0.23. Thanks for the work, Daryn.
          Hide
          Kihwal Lee added a comment -

          +1 for the branch-0.23 patch.

          Show
          Kihwal Lee added a comment - +1 for the branch-0.23 patch.
          Hide
          Daryn Sharp added a comment -

          I ran the webhdfs tests, and as cited above tested that a 401 does not cause a NPE.

          Show
          Daryn Sharp added a comment - I ran the webhdfs tests, and as cited above tested that a 401 does not cause a NPE.
          Hide
          Kihwal Lee added a comment -

          The patch looks okay. Full test-patch won't be necessary, if you have full hdfs test result + manual test result of the fix.

          Show
          Kihwal Lee added a comment - The patch looks okay. Full test-patch won't be necessary, if you have full hdfs test result + manual test result of the fix.
          Hide
          Kihwal Lee added a comment -

          Can you run test-patch? Or at least share your test results?

          Show
          Kihwal Lee added a comment - Can you run test-patch? Or at least share your test results?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12572111/HDFS-4553.branch-23.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4040//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12572111/HDFS-4553.branch-23.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4040//console This message is automatically generated.
          Hide
          Daryn Sharp added a comment -

          Patch is 23-only, so pre-commit will fail to apply.

          Show
          Daryn Sharp added a comment - Patch is 23-only, so pre-commit will fail to apply.
          Hide
          Daryn Sharp added a comment -

          Issue already fixed on trunk via HDFS-3667 that added retry support to webhdfs. The fix was extraneous to the retry changes, so I've extracted the couple of relevant lines.

          No test is included. It's a simple change, HDFS-3667 included no test, and it sure doesn't look simple to test.

          I have manually verified it on a secure cluster. The issue was originally uncovered by a 401 Unauthorized response. It now handles that case correctly.

          Show
          Daryn Sharp added a comment - Issue already fixed on trunk via HDFS-3667 that added retry support to webhdfs. The fix was extraneous to the retry changes, so I've extracted the couple of relevant lines. No test is included. It's a simple change, HDFS-3667 included no test, and it sure doesn't look simple to test. I have manually verified it on a secure cluster. The issue was originally uncovered by a 401 Unauthorized response. It now handles that case correctly.

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development