Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2071

Use of isConnected() in DataXceiver is invalid

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.22.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The use of Socket.isConnected() in DataXceiver.run() is not valid. It returns false until the connection is made and then always returns true after that. It will never return false after the initial connection is successfully made. Socket.isClosed() or SocketChannel.isOpen() should be used instead, assuming someone is handling SocketException and does Socket.close() or SocketChannel.close(). It seems the op handlers in DataXceiver are diligently using IOUtils.closeStream(), which will invoke SocketChannel.close().

      - } while (s.isConnected() && socketKeepaliveTimeout > 0);
      + } while (!s.isClosed() && socketKeepaliveTimeout > 0);
      

      The effect of this bug is very minor, as the socket is read again right after. If the connection was closed, the readOp() will throw an EOFException, which is caught and dealt with properly. The system still functions normally with probably only few microseconds of extra overhead in the premature connection closure cases.

      1. HDFS-2071.patch
        0.7 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          Kihwal Lee added a comment -

          The bug introduced in HDFS-941

          Show
          Kihwal Lee added a comment - The bug introduced in HDFS-941
          Hide
          Kihwal Lee added a comment -

          The system functions without patch with almost no noticeable timing difference.
          No test is added since the correctness cannot be checked by running tests.

          Show
          Kihwal Lee added a comment - The system functions without patch with almost no noticeable timing difference. No test is added since the correctness cannot be checked by running tests.
          Hide
          Harsh J added a comment -

          Kihwal - Since HDFS-941 is not yet committed, I think you can comment/review on that JIRA regarding this bug. Would be worth putting into as a single commit than having this follow up to that?

          Show
          Harsh J added a comment - Kihwal - Since HDFS-941 is not yet committed, I think you can comment/review on that JIRA regarding this bug. Would be worth putting into as a single commit than having this follow up to that?
          Hide
          Kihwal Lee added a comment -

          HDFS-941 is already in trunk. The current discussion mainly for 0.22.

          Show
          Kihwal Lee added a comment - HDFS-941 is already in trunk. The current discussion mainly for 0.22.
          Hide
          Harsh J added a comment -

          Gah, missed that, and the comments there. I take back my suggestion

          Show
          Harsh J added a comment - Gah, missed that, and the comments there. I take back my suggestion
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12482696/HDFS-2071.patch
          against trunk revision 1136132.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.TestHDFSTrash

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/787//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/787//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/787//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12482696/HDFS-2071.patch against trunk revision 1136132. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestHDFSTrash +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/787//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/787//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/787//console This message is automatically generated.
          Hide
          Kihwal Lee added a comment -

          Test failures:

          Tests included: No test is included as justified above.

          Show
          Kihwal Lee added a comment - Test failures: TestHDFSCLI: The quota related test failures are not due to this patch. They also failed in build #696. TestHDFSTrash : It was failing in other recent pre-commit builds: e.g. https://builds.apache.org/job/PreCommit-HDFS-Build/786/ Tests included: No test is included as justified above.
          Hide
          Todd Lipcon added a comment -

          +1, committed. Thanks Kihwal

          Show
          Todd Lipcon added a comment - +1, committed. Thanks Kihwal
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #746 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #70 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/70/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #70 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/70/ )

            People

            • Assignee:
              Kihwal Lee
              Reporter:
              Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development