Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8797

WebHdfsFileSystem creates too many connections for pread

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While running a test we found that WebHdfsFileSystem can create several thousand connections when doing a position read of a 200MB file. For each connection the client will connect to the DataNode again and the DataNode will create a new DFSClient instance to handle the read request. This also leads to several thousand getBlockLocations call to the NameNode.

      The cause of the issue is that in FSInputStream#read(long, byte[], int, int), each time the inputstream reads some time, it seeks back to the old position and resets its state to SEEK. Thus the next read will regenerate the connection.

        public int read(long position, byte[] buffer, int offset, int length)
          throws IOException {
          synchronized (this) {
            long oldPos = getPos();
            int nread = -1;
            try {
              seek(position);
              nread = read(buffer, offset, length);
            } finally {
              seek(oldPos);
            }
            return nread;
          }
        }
      
      1. HDFS-8797.000.patch
        5 kB
        Jing Zhao
      2. HDFS-8797.001.patch
        8 kB
        Jing Zhao
      3. HDFS-8797.002.patch
        11 kB
        Jing Zhao
      4. HDFS-8797.003.patch
        11 kB
        Jing Zhao

        Activity

        Hide
        shv Konstantin Shvachko added a comment -

        Pushed to branch-2.7. Only a minor conflict in TestWebHDFS.
        Updated Fix versions.

        Show
        shv Konstantin Shvachko added a comment - Pushed to branch-2.7. Only a minor conflict in TestWebHDFS. Updated Fix versions.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2211 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2211/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2211 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2211/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/262/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/262/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #254 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/254/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #254 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/254/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2192 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2192/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2192 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2192/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #265 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/265/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #265 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/265/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #995 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/995/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #995 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/995/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        Hide
        jingzhao Jing Zhao added a comment -

        I've committed this to trunk and branch-2. Thanks Nicholas and Yi for the review!

        Show
        jingzhao Jing Zhao added a comment - I've committed this to trunk and branch-2. Thanks Nicholas and Yi for the review!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8203 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8203/)
        HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8203 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8203/ ) HDFS-8797 . WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. (jing9: rev e91ccfad07ec5b5674a84009772dd31a82b4e4de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestByteRangeInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java
        Hide
        hitliuyi Yi Liu added a comment -

        Thanks Jing! New patch looks good to me too.

        Show
        hitliuyi Yi Liu added a comment - Thanks Jing! New patch looks good to me too.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 37s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 36s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 33s The applied patch generated 5 new checkstyle issues (total was 25, now 30).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 23s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 0s Pre-build of native portion
        -1 hdfs tests 160m 24s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 26s Tests passed in hadoop-hdfs-client.
            208m 57s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
          hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746589/HDFS-8797.003.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 4025326
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11790/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11790/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 33s The applied patch generated 5 new checkstyle issues (total was 25, now 30). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 0s Pre-build of native portion -1 hdfs tests 160m 24s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 26s Tests passed in hadoop-hdfs-client.     208m 57s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746589/HDFS-8797.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4025326 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11790/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11790/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11790/console This message was automatically generated.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        +1 the new patch looks good.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - +1 the new patch looks good.
        Hide
        jingzhao Jing Zhao added a comment -

        Yeah, it will also be good to keep read/readFully consistent. Update the patch to override read in ByteRangeInputStream.

        Show
        jingzhao Jing Zhao added a comment - Yeah, it will also be good to keep read/readFully consistent. Update the patch to override read in ByteRangeInputStream .
        Hide
        hitliuyi Yi Liu added a comment -

        readFully call read repeated so that it is a problem. read itself seems fine.
        Yeah, looks like the main issue is with readFully here. So currently I keep the original read unchanged.

        Sorry, I was dizzy here, readFully is the main issue, but is it a bit more efficient if we use the same way for normal pread?

        1. The new approach hear is to open a separate new connection for pread, then close it after finish. When client does stateful read again, original connection is not affected.
        2. seek + read + seek: will close the original connection hold by stateful read, and open a new connection for pread. But when client does stateful read again, connection should be close and open again.

        So #2 (seek + read + seek) requires additional one more close/open connection for normal pread?

        Show
        hitliuyi Yi Liu added a comment - readFully call read repeated so that it is a problem. read itself seems fine. Yeah, looks like the main issue is with readFully here. So currently I keep the original read unchanged. Sorry, I was dizzy here, readFully is the main issue, but is it a bit more efficient if we use the same way for normal pread? The new approach hear is to open a separate new connection for pread, then close it after finish. When client does stateful read again, original connection is not affected. seek + read + seek : will close the original connection hold by stateful read, and open a new connection for pread. But when client does stateful read again, connection should be close and open again. So #2 ( seek + read + seek ) requires additional one more close/open connection for normal pread?
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 43s There were no new javac warning messages.
        +1 javadoc 9m 42s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 42s The applied patch generated 5 new checkstyle issues (total was 25, now 30).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 26s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 4s Pre-build of native portion
        -1 hdfs tests 160m 34s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 27s Tests passed in hadoop-hdfs-client.
            210m 2s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746443/HDFS-8797.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 393fe71
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11779/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11779/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 42s The applied patch generated 5 new checkstyle issues (total was 25, now 30). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 26s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 4s Pre-build of native portion -1 hdfs tests 160m 34s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 27s Tests passed in hadoop-hdfs-client.     210m 2s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746443/HDFS-8797.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 393fe71 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11779/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11779/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11779/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 42s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 36s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 31s The applied patch generated 5 new checkstyle issues (total was 25, now 30).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 24s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 2s Pre-build of native portion
        -1 hdfs tests 160m 23s Tests failed in hadoop-hdfs.
        -1 hdfs tests 0m 29s Tests failed in hadoop-hdfs-client.
            209m 6s  



        Reason Tests
        Failed unit tests hadoop.hdfs.tools.TestDFSAdminWithHA
          hadoop.hdfs.TestDistributedFileSystem
        Failed build hadoop-hdfs-client



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746434/HDFS-8797.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 393fe71
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11777/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11777/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 42s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 31s The applied patch generated 5 new checkstyle issues (total was 25, now 30). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 24s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 2s Pre-build of native portion -1 hdfs tests 160m 23s Tests failed in hadoop-hdfs. -1 hdfs tests 0m 29s Tests failed in hadoop-hdfs-client.     209m 6s   Reason Tests Failed unit tests hadoop.hdfs.tools.TestDFSAdminWithHA   hadoop.hdfs.TestDistributedFileSystem Failed build hadoop-hdfs-client Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746434/HDFS-8797.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 393fe71 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/diffcheckstylehadoop-hdfs-client.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11777/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11777/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11777/console This message was automatically generated.
        Hide
        jingzhao Jing Zhao added a comment -

        Update the patch to add the check suggested by Nicholas. Also added a unit test.

        Show
        jingzhao Jing Zhao added a comment - Update the patch to add the check suggested by Nicholas. Also added a unit test.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        Just a minor comment: in the new readFully method, how about using the length returned by openInputStream to detect EOF so that it can fail fast?

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - Just a minor comment: in the new readFully method, how about using the length returned by openInputStream to detect EOF so that it can fail fast?
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for the review, Yi and Nicholas! Update the patch to address comments.

        A minor comment, we should also override int read

        readFully call read repeated so that it is a problem. read itself seems fine.

        Yeah, looks like the main issue is with readFully here. So currently I keep the original read unchanged.

        Show
        jingzhao Jing Zhao added a comment - Thanks for the review, Yi and Nicholas! Update the patch to address comments. A minor comment, we should also override int read readFully call read repeated so that it is a problem. read itself seems fine. Yeah, looks like the main issue is with readFully here. So currently I keep the original read unchanged.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        Need to fix fileLength in openInputStream(..). It could be updated incorrectly.

        > ... A minor comment, we should also override int read(long position, ...

        readFully call read repeated so that it is a problem. read itself seems fine.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - Need to fix fileLength in openInputStream(..). It could be updated incorrectly. > ... A minor comment, we should also override int read(long position, ... readFully call read repeated so that it is a problem. read itself seems fine.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 19m 12s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 42s There were no new javac warning messages.
        +1 javadoc 9m 42s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 32s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 26s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 4s Pre-build of native portion
        -1 hdfs tests 160m 33s Tests failed in hadoop-hdfs.
        +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.
            210m 3s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
          hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
          hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12746223/HDFS-8797.000.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / ed01dc7
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/testrun_hadoop-hdfs.txt
        hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11759/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11759/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 12s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 42s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 32s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 26s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 4s Pre-build of native portion -1 hdfs tests 160m 33s Tests failed in hadoop-hdfs. +1 hdfs tests 0m 28s Tests passed in hadoop-hdfs-client.     210m 3s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot   hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12746223/HDFS-8797.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ed01dc7 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/testrun_hadoop-hdfs.txt hadoop-hdfs-client test log https://builds.apache.org/job/PreCommit-HDFS-Build/11759/artifact/patchprocess/testrun_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11759/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11759/console This message was automatically generated.
        Hide
        hitliuyi Yi Liu added a comment -

        Thanks Jing for working on this, I think it's a good approach to override readFully and create a new InputStream for pread. A minor comment, we should also override int read(long position, ...

        Show
        hitliuyi Yi Liu added a comment - Thanks Jing for working on this, I think it's a good approach to override readFully and create a new InputStream for pread. A minor comment, we should also override int read(long position, ...
        Hide
        jingzhao Jing Zhao added a comment -

        One possible way to fix can be to override the readFully method in ByteRangeInputStream in which we use a newly created InputStream so that we do not need to touch the internal states. Upload a patch to demo the idea.

        Show
        jingzhao Jing Zhao added a comment - One possible way to fix can be to override the readFully method in ByteRangeInputStream in which we use a newly created InputStream so that we do not need to touch the internal states. Upload a patch to demo the idea.
        Hide
        jingzhao Jing Zhao added a comment -

        Simply removing "seek(oldPos)" can avoid the issue but also changes the behavior: with resetting the position, after a pread we can continue the original stateful read.

        Show
        jingzhao Jing Zhao added a comment - Simply removing "seek(oldPos)" can avoid the issue but also changes the behavior: with resetting the position, after a pread we can continue the original stateful read.

          People

          • Assignee:
            jingzhao Jing Zhao
            Reporter:
            jingzhao Jing Zhao
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development