Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9092

Nfs silently drops overlapping write requests and causes data copying to fail

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: nfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When NOT using 'sync' option, the NFS writes may issue the following warning:
      org.apache.hadoop.hdfs.nfs.nfs3.OpenFileCtx: Got an overlapping write (1248751616, 1249677312), nextOffset=1248752400. Silently drop it now

      and the size of data copied via NFS will stay at 1248752400.

      Found what happened is:

      1. The write requests from client are sent asynchronously.
      2. The NFS gateway has handler to handle the incoming requests by creating an internal write request structuire and put it into cache;
      3. In parallel, a separate thread in NFS gateway takes requests out from the cache and writes the data to HDFS.

      The current offset is how much data has been written by the write thread in 3. The detection of overlapping write request happens in 2, but it only checks the write request against the curent offset, and trim the request if necessary. Because the write requests are sent asynchronously, if two requests are beyond the current offset, and they overlap, it's not detected and both are put into the cache. This cause the symptom reported in this case at step 3.

      1. HDFS-9092.001.patch
        17 kB
        Yongjun Zhang
      2. HDFS-9092.002.patch
        18 kB
        Yongjun Zhang

        Activity

        Hide
        yzhangal Yongjun Zhang added a comment -

        Uploaded patch rev 001.

        Hi Brandon Li,

        Thanks a lot for some side discussion about this issue earlier. i got to the bottom and implemented a solution. Would you please help taking a look at the patch? Very much appreciated!

        Show
        yzhangal Yongjun Zhang added a comment - Uploaded patch rev 001. Hi Brandon Li , Thanks a lot for some side discussion about this issue earlier. i got to the bottom and implemented a solution. Would you please help taking a look at the patch? Very much appreciated!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 21s Pre-patch trunk compilation is healthy.
        +1 @author 0m 1s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 58s There were no new javac warning messages.
        +1 javadoc 10m 15s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 22s There were no new checkstyle issues.
        -1 whitespace 0m 1s The patch has 18 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        -1 findbugs 0m 53s The patch appears to introduce 2 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 15s Pre-build of native portion
        +1 hdfs tests 1m 46s Tests passed in hadoop-hdfs-nfs.
            43m 23s  



        Reason Tests
        FindBugs module:hadoop-hdfs-nfs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12756410/HDFS-9092.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 0832b38
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/whitespace.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-nfs.html
        hadoop-hdfs-nfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/testrun_hadoop-hdfs-nfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12502/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12502/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 21s Pre-patch trunk compilation is healthy. +1 @author 0m 1s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 15s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 22s There were no new checkstyle issues. -1 whitespace 0m 1s The patch has 18 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 0m 53s The patch appears to introduce 2 new Findbugs (version 3.0.0) warnings. +1 native 3m 15s Pre-build of native portion +1 hdfs tests 1m 46s Tests passed in hadoop-hdfs-nfs.     43m 23s   Reason Tests FindBugs module:hadoop-hdfs-nfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756410/HDFS-9092.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0832b38 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-nfs.html hadoop-hdfs-nfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12502/artifact/patchprocess/testrun_hadoop-hdfs-nfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12502/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12502/console This message was automatically generated.
        Hide
        brandonli Brandon Li added a comment - - edited

        Thank you, Yongjun Zhang for the patch. Could you roughly describe the idea of the fix? possibly by copy&paste the comment from the code to here.

        Show
        brandonli Brandon Li added a comment - - edited Thank you, Yongjun Zhang for the patch. Could you roughly describe the idea of the fix? possibly by copy&paste the comment from the code to here.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Brandon Li.

        Good idea, below is some comment I made in the code, hope it helps:

        /**

        • Overlapping Write Request Handling
        • A write request can be in three states:
        • s0. just created, with data != null
        • s1. dumped as length "count", and data set to null
        • s2. read back from dumped area as length "count"
        • Write requests may have overlapping range, we detect this by comparing
        • the data offset range of the request against the current offset of data
        • already written to HDFS. There are two categories:
        • 1. If the beginning part of a new write request data is already written
        • due to an earlier request, we alter the new request by trimming this
        • portion before the new request enters state s0, and the originalCount is
        • remembered.
        • 2. If the lower end of the write request range is beyond the current
        • offset of data already written, we put the request into cache, and detect
        • the overlapping when taking the request out from cache.
        • For category 2, if we find out that a write request overlap with another,
        • this write request is already in state s0, s1, or s3. We trim the
        • beginning part of this request, by remembering the size of this portion
        • as trimDelta. So the resulted offset of the write request is
        • "offset + trimDelta" and the resulted size of the write request is
        • "count - trimDelta".
        • What important to notice is, if the request is in s1 when we do the
        • trimming, the data dumped is of size "count", so when we load
        • the data back from dumped area, we should set the position of the data
        • buffer to trimDelta.
          */
        Show
        yzhangal Yongjun Zhang added a comment - Thanks Brandon Li . Good idea, below is some comment I made in the code, hope it helps: /** Overlapping Write Request Handling A write request can be in three states: s0. just created, with data != null s1. dumped as length "count", and data set to null s2. read back from dumped area as length "count" Write requests may have overlapping range, we detect this by comparing the data offset range of the request against the current offset of data already written to HDFS. There are two categories: 1. If the beginning part of a new write request data is already written due to an earlier request, we alter the new request by trimming this portion before the new request enters state s0, and the originalCount is remembered. 2. If the lower end of the write request range is beyond the current offset of data already written, we put the request into cache, and detect the overlapping when taking the request out from cache. For category 2, if we find out that a write request overlap with another, this write request is already in state s0, s1, or s3. We trim the beginning part of this request, by remembering the size of this portion as trimDelta. So the resulted offset of the write request is "offset + trimDelta" and the resulted size of the write request is "count - trimDelta". What important to notice is, if the request is in s1 when we do the trimming, the data dumped is of size "count", so when we load the data back from dumped area, we should set the position of the data buffer to trimDelta. */
        Hide
        yzhangal Yongjun Zhang added a comment -

        BTW,

        Category 1 is handled by existing code (however, it will not add the write request to cache if the cache is not empty). category 2 is where the issue reported in this jira from. My fix handles the case when cache is not empty for category 1, and handles category 2 as described above.

        Show
        yzhangal Yongjun Zhang added a comment - BTW, Category 1 is handled by existing code (however, it will not add the write request to cache if the cache is not empty). category 2 is where the issue reported in this jira from. My fix handles the case when cache is not empty for category 1, and handles category 2 as described above.
        Hide
        brandonli Brandon Li added a comment -

        For category2, the assumption of the fix is that, the trimmed data is the same as what's already written to HDFS.

        So far we claim that NFS gateway supports the user cases of file uploading and file streaming.
        For file uploading, the overlapped section is safe to drop since it will be the same as what's already written to HDFS. It's the same case for file streaming.

        The only possible problem is this: before the patch, if users do random update to an HDFS file, NFS gateway will report error. With this patch, there is a chance we wont's see the error if it happens that the updated section is trimmed.

        Since random write is not supported anyway, the possible nicer reaction to a random update seems still acceptable to me.

        Show
        brandonli Brandon Li added a comment - For category2, the assumption of the fix is that, the trimmed data is the same as what's already written to HDFS. So far we claim that NFS gateway supports the user cases of file uploading and file streaming. For file uploading, the overlapped section is safe to drop since it will be the same as what's already written to HDFS. It's the same case for file streaming. The only possible problem is this: before the patch, if users do random update to an HDFS file, NFS gateway will report error. With this patch, there is a chance we wont's see the error if it happens that the updated section is trimmed. Since random write is not supported anyway, the possible nicer reaction to a random update seems still acceptable to me.
        Hide
        brandonli Brandon Li added a comment -

        +1. Patch looks good to me. Thank you Yongjun Zhang

        Show
        brandonli Brandon Li added a comment - +1. Patch looks good to me. Thank you Yongjun Zhang
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks a lot Brandon Li!

        I found trunk is changed such that the patch no longer compiles. Uploading rev 002 to address it. In addition, removed extra spaces reported by last jenkins job.

        I originally planned to get this into 2.6.2, however, I found that both 2.7 and 2.6 miss some changes thus the patch can not be applied cleanly. I'm targetting this change to 2.8 for now.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks a lot Brandon Li ! I found trunk is changed such that the patch no longer compiles. Uploading rev 002 to address it. In addition, removed extra spaces reported by last jenkins job. I originally planned to get this into 2.6.2, however, I found that both 2.7 and 2.6 miss some changes thus the patch can not be applied cleanly. I'm targetting this change to 2.8 for now.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 38s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 10m 23s There were no new javadoc warning messages.
        -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
        +1 checkstyle 1m 47s There were no new checkstyle issues.
        -1 whitespace 0m 1s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        -1 findbugs 3m 23s The patch appears to introduce 2 new Findbugs (version 3.0.0) warnings.
        +1 native 3m 13s Pre-build of native portion
        -1 hdfs tests 162m 40s Tests failed in hadoop-hdfs.
        +1 hdfs tests 1m 47s Tests passed in hadoop-hdfs-nfs.
            212m 19s  



        Reason Tests
        FindBugs module:hadoop-hdfs-nfs
        Failed unit tests hadoop.hdfs.server.namenode.TestStartupProgressServlet
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.tools.TestDFSAdminWithHA
          hadoop.hdfs.web.TestWebHDFSOAuth2
          hadoop.hdfs.server.namenode.TestCacheDirectives



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764084/HDFS-9092.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e5992ef
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/patchReleaseAuditProblems.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/whitespace.txt
        Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-nfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-nfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/testrun_hadoop-hdfs-nfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12719/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12719/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 38s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 10m 23s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. +1 checkstyle 1m 47s There were no new checkstyle issues. -1 whitespace 0m 1s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 3m 23s The patch appears to introduce 2 new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 162m 40s Tests failed in hadoop-hdfs. +1 hdfs tests 1m 47s Tests passed in hadoop-hdfs-nfs.     212m 19s   Reason Tests FindBugs module:hadoop-hdfs-nfs Failed unit tests hadoop.hdfs.server.namenode.TestStartupProgressServlet   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.hdfs.server.namenode.TestCacheDirectives Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764084/HDFS-9092.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e5992ef Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/patchReleaseAuditProblems.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/whitespace.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-nfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-nfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/testrun_hadoop-hdfs-nfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12719/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12719/console This message was automatically generated.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8537 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8537/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8537 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8537/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Brandon Li a lot for the review!

        I committed to trunk and branch-2.

        BTW, the test audit msg is due to my accidental inclusion of CHANGES.txt in the patch; somehow my earlier cleaning of whitespaces using --whitespace=fix did not clean up everything; the new test failure is irrelevant to the fix here.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Brandon Li a lot for the review! I committed to trunk and branch-2. BTW, the test audit msg is due to my accidental inclusion of CHANGES.txt in the patch; somehow my earlier cleaning of whitespaces using --whitespace=fix did not clean up everything; the new test failure is irrelevant to the fix here.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #464 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/464/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #464 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/464/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2400 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2400/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2400 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2400/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1195/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1195/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2372 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2372/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2372 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2372/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #457 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/457/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #457 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/457/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #432 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/432/)
        HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec)

        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java
        • hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #432 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/432/ ) HDFS-9092 . Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. (yzhang: rev 151fca5032719e561226ef278e002739073c23ec) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/WriteCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OpenFileCtx.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/OffsetRange.java hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/nfs3/TestWrites.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        liuml07 Mingliang Liu added a comment -

        There are new findbugs warnings in the last patch. Please comment in HDFS-9245 that the findbugs warnings can be ignored.

        Show
        liuml07 Mingliang Liu added a comment - There are new findbugs warnings in the last patch. Please comment in HDFS-9245 that the findbugs warnings can be ignored.
        Show
        yzhangal Yongjun Zhang added a comment - Hi Mingliang Liu , Per https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs-nfs.html It shows 0 findbugs. Guess you meant Pre-patch Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/12719/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html Thanks.
        Hide
        liuml07 Mingliang Liu added a comment -

        Thank you Yongjun Zhang for your quick reply. You're right. The Jenkins report shows 0 findbugs. I can reproduce the findbugs warnings locally and saw the pre-patch in the QA report.

        Show
        liuml07 Mingliang Liu added a comment - Thank you Yongjun Zhang for your quick reply. You're right. The Jenkins report shows 0 findbugs. I can reproduce the findbugs warnings locally and saw the pre-patch in the QA report.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Mingliang Liu. It'd be nice to find out why jenkins job did not report the same. We should look into HDFS-9245 too.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Mingliang Liu . It'd be nice to find out why jenkins job did not report the same. We should look into HDFS-9245 too.
        Hide
        liuml07 Mingliang Liu added a comment -

        It happens to me sometimes when the Jenkins did not report findbugs, in which case I have to run it locally to double check. It will be nice if we can find out why.

        As to the warning itself, I think the unsynchronized access is read only, and for LOG/toString purpose. Do it make sense to make the offset and originalCount volatile?

        Show
        liuml07 Mingliang Liu added a comment - It happens to me sometimes when the Jenkins did not report findbugs, in which case I have to run it locally to double check. It will be nice if we can find out why. As to the warning itself, I think the unsynchronized access is read only, and for LOG/toString purpose. Do it make sense to make the offset and originalCount volatile?
        Hide
        liuml07 Mingliang Liu added a comment -

        s/Do/Does/

        Show
        liuml07 Mingliang Liu added a comment - s/Do/Does/

          People

          • Assignee:
            yzhangal Yongjun Zhang
            Reporter:
            yzhangal Yongjun Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development