Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6097

zero-copy reads are incorrectly disabled on file offsets above 2GB

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.4.0
    • hdfs-client
    • None

    Description

      Zero-copy reads are incorrectly disabled on file offsets above 2GB due to some code that is supposed to disable zero-copy reads on offsets in block files greater than 2GB (because MappedByteBuffer segments are limited to that size).

      Attachments

        1. HDFS-6097.003.patch
          10 kB
          Colin McCabe
        2. HDFS-6097.004.patch
          10 kB
          Colin McCabe
        3. HDFS-6097.005.patch
          11 kB
          Colin McCabe

        Activity

          cmccabe Colin McCabe added a comment -

          fix typo

          cmccabe Colin McCabe added a comment - fix typo
          cmccabe Colin McCabe added a comment -

          fix log message

          cmccabe Colin McCabe added a comment - fix log message
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12634324/HDFS-6097.003.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6389//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6389//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634324/HDFS-6097.003.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6389//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6389//console This message is automatically generated.
          cnauroth Chris Nauroth added a comment -

          The patch looks good, Colin. Just a few small things:

          1. DFSInputStream#tryReadZeroCopy: It seems unnecessary to copy pos to curPos. The value of curPos is never changed throughout the method, so it's always the same as pos. This is a synchronized method, so I don't expect pos to get mutated on a different thread.
          2. TestEnhancedByteBufferAccess: Let's remove the commented out lines and the extra indentation on the Assert.fail line. Let's use try-finally blocks to guarantee cleanup of cluster, fs, fsIn and fsIn2.
          cnauroth Chris Nauroth added a comment - The patch looks good, Colin. Just a few small things: DFSInputStream#tryReadZeroCopy : It seems unnecessary to copy pos to curPos . The value of curPos is never changed throughout the method, so it's always the same as pos . This is a synchronized method, so I don't expect pos to get mutated on a different thread. TestEnhancedByteBufferAccess : Let's remove the commented out lines and the extra indentation on the Assert.fail line. Let's use try-finally blocks to guarantee cleanup of cluster , fs , fsIn and fsIn2 .
          cmccabe Colin McCabe added a comment -

          DFSInputStream#tryReadZeroCopy: It seems unnecessary to copy pos to curPos. The value of curPos is never changed throughout the method, so it's always the same as pos. This is a synchronized method, so I don't expect pos to get mutated on a different thread.

          This is actually an optimization I made. I wanted to avoid making a memory access each time, since I use this variable a lot. By copying it to a local variable, it becomes a lot more obvious to the optimizer that it can't change. It's possible that Java will perform this optimization automatically, but I'm skeptical because we're calling a lot of functions here. It seems like it would require a sophisticated optimizer to realize that there was no code path that changed this variable.

          TestEnhancedByteBufferAccess: Let's remove the commented out lines and the extra indentation on the Assert.fail line.

          OK.

          Let's use try-finally blocks to guarantee cleanup of cluster, fs, fsIn and fsIn2.

          I guess I've started to skip doing this on unit tests. My rationale is that if the test fails, cleanup isn't really that important (the surefire process will simply terminate). In the meantime, try... finally blocks complicate the code and often make it hard to see where a test originally failed. Oftentimes if things get messed up, FileSystem#close or MiniDFSCluster#shutdown will throw an exception. And you end up seeing this unhelpful exception rather than the root cause of the problem displayed in the maven test output. On the other hand, I suppose going without try... finally could encourage people to copy flawed code, so I guess that's the counter-argument.

          cmccabe Colin McCabe added a comment - DFSInputStream#tryReadZeroCopy: It seems unnecessary to copy pos to curPos. The value of curPos is never changed throughout the method, so it's always the same as pos. This is a synchronized method, so I don't expect pos to get mutated on a different thread. This is actually an optimization I made. I wanted to avoid making a memory access each time, since I use this variable a lot. By copying it to a local variable, it becomes a lot more obvious to the optimizer that it can't change. It's possible that Java will perform this optimization automatically, but I'm skeptical because we're calling a lot of functions here. It seems like it would require a sophisticated optimizer to realize that there was no code path that changed this variable. TestEnhancedByteBufferAccess: Let's remove the commented out lines and the extra indentation on the Assert.fail line. OK. Let's use try-finally blocks to guarantee cleanup of cluster, fs, fsIn and fsIn2. I guess I've started to skip doing this on unit tests. My rationale is that if the test fails, cleanup isn't really that important (the surefire process will simply terminate). In the meantime, try... finally blocks complicate the code and often make it hard to see where a test originally failed. Oftentimes if things get messed up, FileSystem#close or MiniDFSCluster#shutdown will throw an exception. And you end up seeing this unhelpful exception rather than the root cause of the problem displayed in the maven test output. On the other hand, I suppose going without try... finally could encourage people to copy flawed code, so I guess that's the counter-argument.
          cmccabe Colin McCabe added a comment -

          Thanks for the review, Chris. I'm going to put out a new version in a sec with the test cleanups, and with try... finally in the test.

          I guess I'll bring up the try... finally issue on the mailing list at some point, and see what people think. In the meantime, I'd like to get this in soon so we can continue testing...

          cmccabe Colin McCabe added a comment - Thanks for the review, Chris. I'm going to put out a new version in a sec with the test cleanups, and with try... finally in the test. I guess I'll bring up the try... finally issue on the mailing list at some point, and see what people think. In the meantime, I'd like to get this in soon so we can continue testing...
          cnauroth Chris Nauroth added a comment -

          This is actually an optimization I made.

          I see. Thanks for explaining. Would you mind putting a comment in there?

          I guess I've started to skip doing this on unit tests.

          I got into the try-finally habit during the Windows work. On Windows, we'd have one test fail and leave the cluster running, because it wasn't doing shutdown. Then, subsequent tests also would fail during initialization due to the more pessimistic file locking behavior on Windows. The prior cluster still held locks on the test data directory, so the subsequent tests couldn't reformat. The subsequent tests would have passed otherwise, so this had the effect of disrupting full test run reports with a lot of false failures. It made it more difficult to determine exactly which test was really failing.

          If the stack traces from close aren't helpful, then we can stifle them by calling IOUtils#cleanup and passing a null logger.

          FWIW, my current favorite way to do this is cluster initialization in a BeforeClass method, cluster shutdown in an AfterClass method, and sometimes close of individual streams or file systems in an After method depending on what the test is doing. This reigns in the code clutter of try-finally. It's not always convenient though if you need to change Configuration in each test or if you need per-test isolation for some other reason.

          cnauroth Chris Nauroth added a comment - This is actually an optimization I made. I see. Thanks for explaining. Would you mind putting a comment in there? I guess I've started to skip doing this on unit tests. I got into the try-finally habit during the Windows work. On Windows, we'd have one test fail and leave the cluster running, because it wasn't doing shutdown. Then, subsequent tests also would fail during initialization due to the more pessimistic file locking behavior on Windows. The prior cluster still held locks on the test data directory, so the subsequent tests couldn't reformat. The subsequent tests would have passed otherwise, so this had the effect of disrupting full test run reports with a lot of false failures. It made it more difficult to determine exactly which test was really failing. If the stack traces from close aren't helpful, then we can stifle them by calling IOUtils#cleanup and passing a null logger. FWIW, my current favorite way to do this is cluster initialization in a BeforeClass method, cluster shutdown in an AfterClass method, and sometimes close of individual streams or file systems in an After method depending on what the test is doing. This reigns in the code clutter of try-finally. It's not always convenient though if you need to change Configuration in each test or if you need per-test isolation for some other reason.
          cnauroth Chris Nauroth added a comment -

          Thanks, Colin. I also meant to add that it's a bit less relevant in this patch, because we know this test won't run on Windows (at least not yet), but like you said it does set a precedent that someone could copy-paste into future tests.

          cnauroth Chris Nauroth added a comment - Thanks, Colin. I also meant to add that it's a bit less relevant in this patch, because we know this test won't run on Windows (at least not yet), but like you said it does set a precedent that someone could copy-paste into future tests.
          cnauroth Chris Nauroth added a comment -

          Thanks for posting v4. Were you also going to put in the comment that copying pos to curPos is an optimization? +1 after that, pending Jenkins run.

          cnauroth Chris Nauroth added a comment - Thanks for posting v4. Were you also going to put in the comment that copying pos to curPos is an optimization? +1 after that, pending Jenkins run.
          cmccabe Colin McCabe added a comment -
          • Add a comment about the curPos optimization
          • add a few more comments to tryReadZeroCopy
          cmccabe Colin McCabe added a comment - Add a comment about the curPos optimization add a few more comments to tryReadZeroCopy
          cnauroth Chris Nauroth added a comment -

          +1 for v5 pending Jenkins. Thanks again, Colin.

          cnauroth Chris Nauroth added a comment - +1 for v5 pending Jenkins. Thanks again, Colin.
          cmccabe Colin McCabe added a comment -

          [try-finally]

          That's a good point, I guess. I had been assuming that the cleanup wasn't really required after a test failure, but that might not be a good assumption. In particular, we'd like to know if the subsequent tests succeeded or failed...

          FWIW, my current favorite way to do this is cluster initialization in a BeforeClass method, cluster shutdown in an AfterClass method, and sometimes close of individual streams or file systems in an After method depending on what the test is doing. This reigns in the code clutter of try-finally. It's not always convenient though if you need to change Configuration in each test or if you need per-test isolation for some other reason.

          It does feel natural to use the Before method, but it also can be inflexible, like you mentioned. I think on balance I usually prefer creating a common function or class that I can have several test functions share. But it does require a try... finally and some extra boilerplate. I wish there were a way to make Before methods apply to only some test methods, or at least modify the configuration they use.

          Thanks for posting v4. Were you also going to put in the comment that copying pos to curPos is an optimization? +1 after that, pending Jenkins run.

          added, thanks

          cmccabe Colin McCabe added a comment - [try-finally] That's a good point, I guess. I had been assuming that the cleanup wasn't really required after a test failure, but that might not be a good assumption. In particular, we'd like to know if the subsequent tests succeeded or failed... FWIW, my current favorite way to do this is cluster initialization in a BeforeClass method, cluster shutdown in an AfterClass method, and sometimes close of individual streams or file systems in an After method depending on what the test is doing. This reigns in the code clutter of try-finally. It's not always convenient though if you need to change Configuration in each test or if you need per-test isolation for some other reason. It does feel natural to use the Before method, but it also can be inflexible, like you mentioned. I think on balance I usually prefer creating a common function or class that I can have several test functions share. But it does require a try... finally and some extra boilerplate. I wish there were a way to make Before methods apply to only some test methods, or at least modify the configuration they use. Thanks for posting v4. Were you also going to put in the comment that copying pos to curPos is an optimization? +1 after that, pending Jenkins run. added, thanks
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12634502/HDFS-6097.004.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6392//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6392//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634502/HDFS-6097.004.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6392//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6392//console This message is automatically generated.
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12634503/HDFS-6097.004.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6393//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6393//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634503/HDFS-6097.004.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6393//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6393//console This message is automatically generated.
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12634509/HDFS-6097.005.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6394//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6394//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12634509/HDFS-6097.005.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6394//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6394//console This message is automatically generated.
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5324 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5324/)
          HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5324 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5324/ ) HDFS-6097 . Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #509 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/509/)
          HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #509 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/509/ ) HDFS-6097 . Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1701 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1701/)
          HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1701 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1701/ ) HDFS-6097 . Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1726 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1726/)
          HDFS-6097. Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1726 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1726/ ) HDFS-6097 . Zero-copy reads are incorrectly disabled on file offsets above 2GB (cmccabe) (cmccabe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1577350 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/ShortCircuitReplica.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestEnhancedByteBufferAccess.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/BlockReaderTestUtil.java

          People

            cmccabe Colin McCabe
            cmccabe Colin McCabe
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: