Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-483

Data transfer (aka pipeline) implementation cannot tolerate exceptions

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: datanode, hdfs-client
    • Labels:
      None

      Description

      Data transfer was tested with simulated exceptions as below:

      1. create files with dfs
      2. write 1 byte
      3. close file
      4. open the same file
      5. read the 1 byte and compare results

      The file was closed successfully but we got an IOException(Could not get block locations...) when the file was reopened for reading.

      1. h483_20090819b.patch
        31 kB
        Tsz Wo Nicholas Sze
      2. h483_20090819.patch
        27 kB
        Tsz Wo Nicholas Sze
      3. h483_20090818.patch
        36 kB
        Tsz Wo Nicholas Sze
      4. h483_20090810.patch
        17 kB
        Tsz Wo Nicholas Sze
      5. h483_20090807b.patch
        8 kB
        Tsz Wo Nicholas Sze
      6. h483_20090807.patch
        32 kB
        Tsz Wo Nicholas Sze
      7. h483_20090806.patch
        27 kB
        Tsz Wo Nicholas Sze
      8. h483_20090731.patch
        24 kB
        Tsz Wo Nicholas Sze
      9. h483_20090730.patch
        20 kB
        Tsz Wo Nicholas Sze
      10. h483_20090727.patch
        17 kB
        Tsz Wo Nicholas Sze
      11. h483_20090717.patch
        17 kB
        Tsz Wo Nicholas Sze
      12. h483_20090713.patch
        18 kB
        Tsz Wo Nicholas Sze
      13. h483_20090709.patch
        15 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          Yes, we should close this and HDFS-264.

          Show
          Tsz Wo Nicholas Sze added a comment - Yes, we should close this and HDFS-264 .
          Hide
          Hairong Kuang added a comment -

          Nicholas, since all sub tasks are resolved, can we close this? Did HDFS-101 also fix HDFS-264?

          Show
          Hairong Kuang added a comment - Nicholas, since all sub tasks are resolved, can we close this? Did HDFS-101 also fix HDFS-264 ?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090819b.patch: added Pipeline_Fi_31 - 35.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090819b.patch: added Pipeline_Fi_31 - 35.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090819.patch: added two more tests and updated with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090819.patch: added two more tests and updated with trunk.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I changed TestFiDataTransferProtocol to junit 4 in HDFS-552. Thanks, Kan and Cos for the advises.

          Show
          Tsz Wo Nicholas Sze added a comment - I changed TestFiDataTransferProtocol to junit 4 in HDFS-552 . Thanks, Kan and Cos for the advises.
          Hide
          Konstantin Boudnik added a comment -

          I think the rule of the thumb is to stay away from junit.framework imports when developing new tests. We are trying to get away from JUnit3 from now on.

          Show
          Konstantin Boudnik added a comment - I think the rule of the thumb is to stay away from junit.framework imports when developing new tests. We are trying to get away from JUnit3 from now on.
          Hide
          Kan Zhang added a comment -

          it's part of junit4 and has 2 more methods for arrays if i remember correctly.

          Show
          Kan Zhang added a comment - it's part of junit4 and has 2 more methods for arrays if i remember correctly.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > shouldn't we use "import static org.junit.Assert.*;" instead?

          What's the difference?

          Show
          Tsz Wo Nicholas Sze added a comment - > shouldn't we use "import static org.junit.Assert.*;" instead? What's the difference?
          Hide
          Kan Zhang added a comment -

          shouldn't we use "import static org.junit.Assert.*;" instead?

          Show
          Kan Zhang added a comment - shouldn't we use "import static org.junit.Assert.*;" instead?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090818.patch: a few more tests and synced with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090818.patch: a few more tests and synced with trunk.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Try Hudson again to see whether the failed fault injection tests can fail the build.

          Great! Got a -1 on core tests although the test report does not show any failure.

          Created HDFS-542 for fixing the test report.

          Show
          Tsz Wo Nicholas Sze added a comment - > Try Hudson again to see whether the failed fault injection tests can fail the build. Great! Got a -1 on core tests although the test report does not show any failure. Created HDFS-542 for fixing the test report.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12416138/h483_20090810.patch
          against trunk revision 803337.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12416138/h483_20090810.patch against trunk revision 803337. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/60/console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Try Hudson again to see whether the failed fault injection tests can fail the build.

          Show
          Tsz Wo Nicholas Sze added a comment - Try Hudson again to see whether the failed fault injection tests can fail the build.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090810.patch: added pipeline_Fi_01,02,03 and changed to junit 4.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090810.patch: added pipeline_Fi_01,02,03 and changed to junit 4.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > -1 core tests. The patch failed core unit tests.

          There are other unit tests failing. So we cannot tells whether the failed fault injection tests can fail the build.

          Show
          Tsz Wo Nicholas Sze added a comment - > -1 core tests. The patch failed core unit tests. There are other unit tests failing. So we cannot tells whether the failed fault injection tests can fail the build.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12415910/h483_20090807b.patch
          against trunk revision 802264.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12415910/h483_20090807b.patch against trunk revision 802264. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-vesta.apache.org/51/console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Submitting for testing Hudson. Check whether the failed fault injection tests can fail the build.

          Show
          Tsz Wo Nicholas Sze added a comment - Submitting for testing Hudson. Check whether the failed fault injection tests can fail the build.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090807b.patch: updated with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090807b.patch: updated with trunk.
          Hide
          Konstantin Boudnik added a comment -

          Would it be possible to make DataNodeAction's constructor to be at least protected so one can extends this class from another package's classes?

          Show
          Konstantin Boudnik added a comment - Would it be possible to make DataNodeAction's constructor to be at least protected so one can extends this class from another package's classes?
          Hide
          Konstantin Boudnik added a comment -

          May I suggest to use JUnit4 notations (e.g. @Test and org.junit imports) instead of JUnit3 (inheritance of TestCase and test methods naming convention)?

          Show
          Konstantin Boudnik added a comment - May I suggest to use JUnit4 notations (e.g. @Test and org.junit imports) instead of JUnit3 (inheritance of TestCase and test methods naming convention)?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          "h483_20090709.patch" should be "h483_20090807.patch"

          Show
          Tsz Wo Nicholas Sze added a comment - "h483_20090709.patch" should be "h483_20090807.patch"
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090709.patch: moved the failed tests to TestFiDataTransferProtocol2.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090709.patch: moved the failed tests to TestFiDataTransferProtocol2.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Do you plan to split the current patch to include only failing test here? ...
          Yes, I will upload a new patch once HDFS-451 is committed.

          Show
          Tsz Wo Nicholas Sze added a comment - > Do you plan to split the current patch to include only failing test here? ... Yes, I will upload a new patch once HDFS-451 is committed.
          Hide
          Konstantin Boudnik added a comment -

          Do you plan to split the current patch to include only failing test here? 'cause otherwise it will render a conflict IMO

          Show
          Konstantin Boudnik added a comment - Do you plan to split the current patch to include only failing test here? 'cause otherwise it will render a conflict IMO
          Hide
          Tsz Wo Nicholas Sze added a comment -

          It turns out that trunk can pass Pipeline_Fi_06,07,12,14,15 among all the implemented tests. I will post a separate patch, which includes the passed tests, in HDFS-451.

          Show
          Tsz Wo Nicholas Sze added a comment - It turns out that trunk can pass Pipeline_Fi_06,07,12,14,15 among all the implemented tests. I will post a separate patch, which includes the passed tests, in HDFS-451 .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090806.patch: refactored the codes and updated with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090806.patch: refactored the codes and updated with trunk.
          Hide
          Konstantin Boudnik added a comment -

          Looks good! Thanks for adding JavaDocs Nicholas.

          Show
          Konstantin Boudnik added a comment - Looks good! Thanks for adding JavaDocs Nicholas.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090731.patch: added javadoc and rewrote some codes for a better style.

          Note that all the posted patches up to now contain only tests (from Pipeline_Fi_04 to Pipeline_Fi_16).

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090731.patch: added javadoc and rewrote some codes for a better style. Note that all the posted patches up to now contain only tests (from Pipeline_Fi_04 to Pipeline_Fi_16).
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090730.patch: added a few more tests for non-responding datanodes and slow datanodes.

          (Cos, will work on the javadoc later.)

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090730.patch: added a few more tests for non-responding datanodes and slow datanodes. (Cos, will work on the javadoc later.)
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090727.patch: updated with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090727.patch: updated with trunk.
          Hide
          Konstantin Boudnik added a comment -

          +1 (conditional to the JavaDoc being added)

          Show
          Konstantin Boudnik added a comment - +1 (conditional to the JavaDoc being added)
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > ... it seems to be the only reference to this member.
          For the index, it is used in run(..) showed below for determining the pipeline position.

          +    public void run(DataNode datanode) {
          +      final Pipeline p = DataTransferTestUtil.getPipeline();
          +      if (p.contains(index, datanode.dnRegistration)) {
          +        throw new OutOfMemoryError("FI: " + currentTest
          +            + ", datanode=" + datanode.dnRegistration.getName());
          +      }
          +    }
          
          Show
          Tsz Wo Nicholas Sze added a comment - > ... it seems to be the only reference to this member. For the index, it is used in run(..) showed below for determining the pipeline position. + public void run(DataNode datanode) { + final Pipeline p = DataTransferTestUtil.getPipeline(); + if (p.contains(index, datanode.dnRegistration)) { + throw new OutOfMemoryError( "FI: " + currentTest + + ", datanode=" + datanode.dnRegistration.getName()); + } + }
          Hide
          Konstantin Boudnik added a comment -

          Although I am not completely sure about the future plans of the development of these tests, but it seems that the second parameter of DoosAction and OomAction hasn't been used by anything but the constructor itself.

          ...
              final int index;
          
               DoosAction(String currentTest, int index) {
                this.currentTest = currentTest;
                this.index = index;
              }
          ...
          

          and it seems to be the only reference to this member.

          Also, it seems that non of newly created public methods/aspects have JavaDocs.

          Show
          Konstantin Boudnik added a comment - Although I am not completely sure about the future plans of the development of these tests, but it seems that the second parameter of DoosAction and OomAction hasn't been used by anything but the constructor itself. ... final int index; DoosAction(String currentTest, int index) { this.currentTest = currentTest; this.index = index; } ... and it seems to be the only reference to this member. Also, it seems that non of newly created public methods/aspects have JavaDocs.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090717.patch: updated with trunk.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090717.patch: updated with trunk.
          Hide
          Konstantin Boudnik added a comment -

          This patch seems to contain some modifications of build.xml file which are part of resolved HDFS-475.

          Show
          Konstantin Boudnik added a comment - This patch seems to contain some modifications of build.xml file which are part of resolved HDFS-475 .
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090713.patch: complete the tests Pipeline_FI_14, 15, 16. Trunk passed Pipeline_FI_14 but failed Pipeline_FI_15, 16.

          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090713.patch: complete the tests Pipeline_FI_14, 15, 16. Trunk passed Pipeline_FI_14 but failed Pipeline_FI_15, 16.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h483_20090709.patch: fault injection tests

          • implements Pipeline_FI_09, 10, 11 described in the test plan posted in HDFS-265.
          • also contains codes for Pipeline_FI_14, 15, 16 but they are not yet completed.
          Show
          Tsz Wo Nicholas Sze added a comment - h483_20090709.patch: fault injection tests implements Pipeline_FI_09, 10, 11 described in the test plan posted in HDFS-265 . also contains codes for Pipeline_FI_14, 15, 16 but they are not yet completed.

            People

            • Assignee:
              Unassigned
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development