Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1843

Discover file not found early for file append

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      I have committed this. Thanks to Bharath!

      Description

      For the append call, discover file not found exception early and avoid extra server call.

      1. HDFS-1843-1.patch
        3 kB
        Bharath Mundlapudi
      2. HDFS-1843-2.patch
        3 kB
        Bharath Mundlapudi

        Activity

        Bharath Mundlapudi created issue -
        Hide
        Bharath Mundlapudi added a comment -

        If getFileInfo returns null then throw FileNotFoundException way early than going through another complete append server call to discover that file was not found. Attached the patch with unit test. Converted the testcase into JUnit 4 version.

        Show
        Bharath Mundlapudi added a comment - If getFileInfo returns null then throw FileNotFoundException way early than going through another complete append server call to discover that file was not found. Attached the patch with unit test. Converted the testcase into JUnit 4 version.
        Bharath Mundlapudi made changes -
        Field Original Value New Value
        Attachment HDFS-1843-1.patch [ 12476518 ]
        Hide
        Jitendra Nath Pandey added a comment -

        1. Please shutdown the cluster at the end of the test.
        2. minor: an extra space in indentation in DFSClient.

        Otherwise the patch looks good. +1

        Show
        Jitendra Nath Pandey added a comment - 1. Please shutdown the cluster at the end of the test. 2. minor: an extra space in indentation in DFSClient. Otherwise the patch looks good. +1
        Hide
        Bharath Mundlapudi added a comment -

        Thanks for code review, Jitendra. I have incorporated the changes.

        Show
        Bharath Mundlapudi added a comment - Thanks for code review, Jitendra. I have incorporated the changes.
        Bharath Mundlapudi made changes -
        Attachment HDFS-1843-2.patch [ 12476943 ]
        Hide
        Jitendra Nath Pandey added a comment -

        +1 for the patch.

        Show
        Jitendra Nath Pandey added a comment - +1 for the patch.
        Jitendra Nath Pandey made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12476943/HDFS-1843-2.patch
        against trunk revision 1096010.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestFileAppend4
        org.apache.hadoop.hdfs.TestFileConcurrentReader
        org.apache.hadoop.hdfs.TestLargeBlock
        org.apache.hadoop.hdfs.TestWriteConfigurationToDFS

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12476943/HDFS-1843-2.patch against trunk revision 1096010. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestFileAppend4 org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.hdfs.TestLargeBlock org.apache.hadoop.hdfs.TestWriteConfigurationToDFS +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/411//console This message is automatically generated.
        Hide
        Bharath Mundlapudi added a comment -

        I have tested on my local setup, the above tests seem to pass except TestLargeBlock and is not related to this patch. TestLargeBlock failed due to "Timeout occurred".

        Show
        Bharath Mundlapudi added a comment - I have tested on my local setup, the above tests seem to pass except TestLargeBlock and is not related to this patch. TestLargeBlock failed due to "Timeout occurred".
        Jitendra Nath Pandey made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note I have committed this. Thanks to Bharath!
        Resolution Fixed [ 1 ]
        Hide
        Jitendra Nath Pandey added a comment -

        I have committed this. Thanks to Bharath!

        Show
        Jitendra Nath Pandey added a comment - I have committed this. Thanks to Bharath!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #606 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/606/)
        HDFS-1843. Discover file not found early for file append. Contributed by Bharath Mundlapudi.

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #606 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/606/ ) HDFS-1843 . Discover file not found early for file append. Contributed by Bharath Mundlapudi.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #608 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/608/)
        Updated CHANGES.txt for my previous commit (HDFS-1843).

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #608 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/608/ ) Updated CHANGES.txt for my previous commit ( HDFS-1843 ).
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #650 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/650/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #650 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/650/ )
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Bharath Mundlapudi
            Reporter:
            Bharath Mundlapudi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development