Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2751

Datanode drops OS cache behind reads even for short reads

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.1, 1.1.0
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-2465 has some code which attempts to disable the "drop cache behind reads" functionality when the reads are <256KB (eg HBase random access). But this check was missing in the close() function, so it always drops cache behind reads regardless of the size of the read. This hurts HBase random read performance when this patch is enabled.

      1. hdfs-2751.txt
        0.8 kB
        Todd Lipcon
      2. hdfs-2751.txt
        0.8 kB
        Todd Lipcon
      3. HDFS-2751.branch-1.patch
        0.7 kB
        Brandon Li

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          (thanks to JD Cryans for finding this one)

          Show
          Todd Lipcon added a comment - (thanks to JD Cryans for finding this one)
          Hide
          Eli Collins added a comment -

          +1 pending jenkins

          Show
          Eli Collins added a comment - +1 pending jenkins
          Hide
          Todd Lipcon added a comment -

          Reattaching same patch to kick jenkins

          Show
          Todd Lipcon added a comment - Reattaching same patch to kick jenkins
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12511219/hdfs-2751.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated 21 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hdfs.TestLeaseRenewer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511219/hdfs-2751.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 21 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestLeaseRenewer +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1788//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Test failure was unrelated. Committed to trunk and 23.

          Show
          Todd Lipcon added a comment - Test failure was unrelated. Committed to trunk and 23.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1560 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1560/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1560 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1560/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #392 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/392/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #392 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/392/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1634 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1634/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1634 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1634/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #383 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/383/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #383 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/383/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #406 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/406/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #406 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/406/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1579 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1579/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1579 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1579/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #931 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/931/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #931 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/931/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #144 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/144/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #144 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/144/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #166 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/166/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #166 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/166/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233795 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #964 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/964/)
          HDFS-2751. Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #964 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/964/ ) HDFS-2751 . Datanode may incorrectly drop OS cache behind reads even for short reads. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1233796 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          Hide
          Brandon Li added a comment -

          Uploaded patch to back port to branch-1.

          Show
          Brandon Li added a comment - Uploaded patch to back port to branch-1.
          Hide
          Brandon Li added a comment -

          I did teragen/terasort/teravalid with the branch-1 patch.

          Show
          Brandon Li added a comment - I did teragen/terasort/teravalid with the branch-1 patch.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 the branch-1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 the branch-1 patch looks good.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed the backport patch to branch-1. Thanks, Brandon!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed the backport patch to branch-1. Thanks, Brandon!
          Hide
          Matt Foley added a comment -

          merged to branch-1.1

          Show
          Matt Foley added a comment - merged to branch-1.1

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development