Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8696

Make the lower and higher watermark in the DN Netty server configurable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: webhdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Introduced two new configuration dfs.webhdfs.netty.low.watermark and dfs.webhdfs.netty.high.watermark to enable tuning the size of the buffers of the Netty server inside Datanodes.

      Description

      There is an issue that appears related to the webhdfs server. When making two concurrent requests, the DN will sometimes pause for extended periods (I've seen 1-300 seconds), killing performance and dropping connections.

      To reproduce:
      1. set up a HDFS cluster
      2. Upload a large file (I was using 10GB). Perform 1-byte reads, writing
      the time out to /tmp/times.txt

      i=1
      while (true); do 
      echo $i
      let i++
      /usr/bin/time -f %e -o /tmp/times.txt -a curl -s -L -o /dev/null "http://<namenode>:50070/webhdfs/v1/tmp/bigfile?op=OPEN&user.name=root&length=1";
      done
      

      3. Watch for 1-byte requests that take more than one second:
      tail -F /tmp/times.txt | grep -E "^[^0]"

      4. After it has had a chance to warm up, start doing large transfers from
      another shell:

      i=1
      while (true); do 
      echo $i
      let i++
      /usr/bin/time -f %e curl -s -L -o /dev/null "http://<namenode>:50070/webhdfs/v1/tmp/bigfile?op=OPEN&user.name=root";
      done
      

      It's easy to find after a minute or two that small reads will sometimes
      pause for 1-300 seconds. In some extreme cases, it appears that the
      transfers timeout and the DN drops the connection.

      1. HDFS-8696.004.patch
        8 kB
        Xiaobing Zhou
      2. HDFS-8696.005.patch
        3 kB
        Bob Hansen
      3. HDFS-8696.006.patch
        3 kB
        Xiaobing Zhou
      4. HDFS-8696.007.patch
        3 kB
        Xiaobing Zhou
      5. HDFS-8696.008.patch
        4 kB
        Xiaobing Zhou
      6. HDFS-8696.009.patch
        4 kB
        Xiaobing Zhou
      7. HDFS-8696.010.patch
        4 kB
        Xiaobing Zhou
      8. HDFS-8696.1.patch
        6 kB
        Xiaobing Zhou
      9. HDFS-8696.2.patch
        5 kB
        Xiaobing Zhou
      10. HDFS-8696.3.patch
        8 kB
        Xiaobing Zhou

        Activity

        Hide
        xiaobingo Xiaobing Zhou added a comment -

        I investigated many aspects of issue, including disk io queue, NN/DN/client packages, no help. it finally boils down to some options we’ve to explicitly set for Netty server.
        1. Netty NIO event loop boss/worker threads, by default they are 1 and 2, which are too small to handle intensive workloads
        2. Though it is platform dependent, ChannelOption.SO_BACKLOG, the maximum queue length for incoming connection, default of which is 50, too small to deal with high concurrency.
        3. These two ChannelOption.SO_SNDBUF and ChannelOption.SO_RCVBUF(sending and receiving buffer size) are the key point to this issue. Performance gains benefit from proper setting significantly. In this case, up to 30 seconds latency will drop to up to 5 in my test env.
        4. ChannelOption.WRITE_BUFFER_LOW_WATER_MARK and ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK(thresholds to indicate channel is writable or not) also matters to drop up to 5 secs latency to less than 1 sec.

        Right now, I am making all these configurable and try best to tune so as to figure out good HDFS default values to be shipped out.

        Show
        xiaobingo Xiaobing Zhou added a comment - I investigated many aspects of issue, including disk io queue, NN/DN/client packages, no help. it finally boils down to some options we’ve to explicitly set for Netty server. 1. Netty NIO event loop boss/worker threads, by default they are 1 and 2, which are too small to handle intensive workloads 2. Though it is platform dependent, ChannelOption.SO_BACKLOG, the maximum queue length for incoming connection, default of which is 50, too small to deal with high concurrency. 3. These two ChannelOption.SO_SNDBUF and ChannelOption.SO_RCVBUF(sending and receiving buffer size) are the key point to this issue. Performance gains benefit from proper setting significantly. In this case, up to 30 seconds latency will drop to up to 5 in my test env. 4. ChannelOption.WRITE_BUFFER_LOW_WATER_MARK and ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK(thresholds to indicate channel is writable or not) also matters to drop up to 5 secs latency to less than 1 sec. Right now, I am making all these configurable and try best to tune so as to figure out good HDFS default values to be shipped out.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Made a patch. Reduced the latency to sub second in my test evn. Can anyone review it? Thanks.

        Show
        xiaobingo Xiaobing Zhou added a comment - Made a patch. Reduced the latency to sub second in my test evn. Can anyone review it? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 19m 5s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 26s There were no new javac warning messages.
        +1 javadoc 10m 22s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 26s The applied patch generated 14 new checkstyle issues (total was 431, now 445).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 42s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        -1 findbugs 3m 9s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken.
        +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 0m 45s Pre-build of native portion
        -1 hdfs tests 166m 32s Tests failed in hadoop-hdfs.
            213m 30s  



        Reason Tests
        Failed unit tests hadoop.fs.TestHdfsNativeCodeLoader



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12743182/HDFS-8696.1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a78d507
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11567/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11567/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 5s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 26s There were no new javac warning messages. +1 javadoc 10m 22s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 26s The applied patch generated 14 new checkstyle issues (total was 431, now 445). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 42s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. -1 findbugs 3m 9s Post-patch findbugs hadoop-hdfs-project/hadoop-hdfs compilation is broken. +1 findbugs 3m 9s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 0m 45s Pre-build of native portion -1 hdfs tests 166m 32s Tests failed in hadoop-hdfs.     213m 30s   Reason Tests Failed unit tests hadoop.fs.TestHdfsNativeCodeLoader Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743182/HDFS-8696.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a78d507 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11567/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11567/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11567/console This message was automatically generated.
        Hide
        bobthansen Bob Hansen added a comment -

        Be sure to set the high water mark before the low water mark. See https://github.com/netty/netty/issues/3806

        Show
        bobthansen Bob Hansen added a comment - Be sure to set the high water mark before the low water mark. See https://github.com/netty/netty/issues/3806
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        patch V2. boss threads setting is unnecessary. Just use default value of 2.

        Show
        xiaobingo Xiaobing Zhou added a comment - patch V2. boss threads setting is unnecessary. Just use default value of 2.
        Hide
        bobthansen Bob Hansen added a comment -

        With the addition of dfs.webhdfs.server.worker.threads, does the hadoop.http.max.threads setting have any effect, or are they different thread pools? Should we be using dfs.webhdfs.server.worker.threads instead of dfs.webhdfs.server.worker.threads?

        Show
        bobthansen Bob Hansen added a comment - With the addition of dfs.webhdfs.server.worker.threads, does the hadoop.http.max.threads setting have any effect, or are they different thread pools? Should we be using dfs.webhdfs.server.worker.threads instead of dfs.webhdfs.server.worker.threads?
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Bob Hansen hadoop.http.max.threads only applies to HttpServer2 which is an embedded Jetty server, having no effects on Netty. I agree to use it so as not to cause confusion.

        Show
        xiaobingo Xiaobing Zhou added a comment - Bob Hansen hadoop.http.max.threads only applies to HttpServer2 which is an embedded Jetty server, having no effects on Netty. I agree to use it so as not to cause confusion.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 17m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 26s There were no new javac warning messages.
        +1 javadoc 9m 39s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 13s The applied patch generated 13 new checkstyle issues (total was 431, now 444).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 3m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 15s Pre-build of native portion
        +1 hdfs tests 162m 23s Tests passed in hadoop-hdfs.
            208m 28s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12743418/HDFS-8696.2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / fc92d3e
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11586/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11586/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 26s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 13s The applied patch generated 13 new checkstyle issues (total was 431, now 444). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 15s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 15s Pre-build of native portion +1 hdfs tests 162m 23s Tests passed in hadoop-hdfs.     208m 28s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743418/HDFS-8696.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fc92d3e Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11586/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11586/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11586/console This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Patch V3, removed dfs.webhdfs.server.worker.threads, keep to use hadoop.http.max.threads.

        Show
        xiaobingo Xiaobing Zhou added a comment - Patch V3, removed dfs.webhdfs.server.worker.threads, keep to use hadoop.http.max.threads.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 21m 8s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 19s There were no new javac warning messages.
        +1 javadoc 10m 16s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 21s The applied patch generated 1 new checkstyle issues (total was 235, now 236).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 48s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        +1 findbugs 5m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 22m 52s Tests failed in hadoop-common.
        -1 hdfs tests 165m 44s Tests failed in hadoop-hdfs.
            239m 26s  



        Reason Tests
        Failed unit tests hadoop.fs.TestLocalFsFCStatistics
          hadoop.hdfs.server.namenode.TestFileTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12743849/HDFS-8696.3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 81f3644
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11593/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11593/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 8s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 19s There were no new javac warning messages. +1 javadoc 10m 16s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 21s The applied patch generated 1 new checkstyle issues (total was 235, now 236). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 48s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 5m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 52s Tests failed in hadoop-common. -1 hdfs tests 165m 44s Tests failed in hadoop-hdfs.     239m 26s   Reason Tests Failed unit tests hadoop.fs.TestLocalFsFCStatistics   hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12743849/HDFS-8696.3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 81f3644 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11593/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11593/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11593/console This message was automatically generated.
        Hide
        Jun_Yin Jun Yin added a comment -

        Hi Xiaobing,

        from my test, we got some unexpected result with a larger file(18G) reading.

        case #1 - unpatched, hdfs-site.xml has following parameters:
        "dfs.webhdfs.server.worker.threads"= 100;
        "dfs.webhdfs.server.max.connection.queue.length" = 1024;
        "dfs.webhdfs.net.send.buf.size" = 65535;
        "dfs.webhdfs.net.receive.buf.size" = 65535;
        "dfs.webhdfs.channel.write.buf.low.watermark" = 65535;
        "dfs.webhdfs.channel.write.buf.high.watermark" = 131070;
        large read test:
        $ while (true); do /usr/bin/time -f %e -o /tmp/times.txt -a curl -s -L -o /dev/null "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release"; done
        $ while (true); do /usr/bin/time -f %e -o /tmp/times.txt -a curl -s -L -o /dev/null "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release&length=1"; done
        $ tail -F /tmp/times.txt | grep -E "^[^0]"
        result:
        according to the /tmp/times.txt, delays are in the range 30-60s

        case #2 - patched, also set required parameters in the config file - hdfs-site.xml
        large read test as same as case #1, result:
        delays are in the range of 40-90s, 2 extremely slow - 155s and 174s

        I will update with some percentile later
        Thanks

        Show
        Jun_Yin Jun Yin added a comment - Hi Xiaobing, from my test, we got some unexpected result with a larger file(18G) reading. case #1 - unpatched, hdfs-site.xml has following parameters: "dfs.webhdfs.server.worker.threads"= 100; "dfs.webhdfs.server.max.connection.queue.length" = 1024; "dfs.webhdfs.net.send.buf.size" = 65535; "dfs.webhdfs.net.receive.buf.size" = 65535; "dfs.webhdfs.channel.write.buf.low.watermark" = 65535; "dfs.webhdfs.channel.write.buf.high.watermark" = 131070; large read test: $ while (true); do /usr/bin/time -f %e -o /tmp/times.txt -a curl -s -L -o /dev/null "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release"; done $ while (true); do /usr/bin/time -f %e -o /tmp/times.txt -a curl -s -L -o /dev/null "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release&length=1"; done $ tail -F /tmp/times.txt | grep -E "^ [^0] " result: according to the /tmp/times.txt, delays are in the range 30-60s case #2 - patched, also set required parameters in the config file - hdfs-site.xml large read test as same as case #1, result: delays are in the range of 40-90s, 2 extremely slow - 155s and 174s I will update with some percentile later Thanks
        Hide
        bobhansen Bob Hansen added a comment -

        Jun - thanks for posting those.

        Can you break down what percentage of requests that were < 50ms, 50-1000ms, +1000ms for the patched and unpatched sets?

        The patched test was with patch v1 (which used the dfs.webhdfs.server.worker.threads setting), correct?

        Show
        bobhansen Bob Hansen added a comment - Jun - thanks for posting those. Can you break down what percentage of requests that were < 50ms, 50-1000ms, +1000ms for the patched and unpatched sets? The patched test was with patch v1 (which used the dfs.webhdfs.server.worker.threads setting), correct?
        Hide
        bobhansen Bob Hansen added a comment -

        Still seeing some periodic slowdowns. Could be related to HDFS-8855; the periodic drop in established connections in that bug may correlate with the periodic jumps in latency we're seeing in this bug.

        Show
        bobhansen Bob Hansen added a comment - Still seeing some periodic slowdowns. Could be related to HDFS-8855 ; the periodic drop in established connections in that bug may correlate with the periodic jumps in latency we're seeing in this bug.
        Hide
        Jun_Yin Jun Yin added a comment -

        good catch, actually we use V3 which Xiaobing sent by email.
        The dfs.webhdfs.server.worker.threads was something taht old patch left in the config file, it has no effect with the latest patch I think.

        Show
        Jun_Yin Jun Yin added a comment - good catch, actually we use V3 which Xiaobing sent by email. The dfs.webhdfs.server.worker.threads was something taht old patch left in the config file, it has no effect with the latest patch I think.
        Hide
        Jun_Yin Jun Yin added a comment -

        On unpatched try
        $ curl -s -L "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release&length=1", will get data which length=1
        While on patched cluster it will get whole data just as same as execution
        $ curl -s -L "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release"
        please try if you can also reproduce, maybe i missed something.

        Show
        Jun_Yin Jun Yin added a comment - On unpatched try $ curl -s -L "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release&length=1", will get data which length=1 While on patched cluster it will get whole data just as same as execution $ curl -s -L "http://NN:50070/webhdfs/v1/tmp/catalog_sales_38_50.dat?op=OPEN&user.name=release" please try if you can also reproduce, maybe i missed something.
        Hide
        bobhansen Bob Hansen added a comment -

        With the patched hdfs, we're still seeing slowdowns.

        For the 1-byte reads:
        99.7% of requests have an average latency of 4.5ms
        0.3% of requests have an average latency of 870 ms (selected for latency > 500ms)

        So we're spending 34% of our wall-clock time waiting for the slow reads. This is better than the unpatched 2.2.6, but still a bit rough.

        Running the same experiment against 2.2.0.0 gave 0 reads at more than 500ms.

        Show
        bobhansen Bob Hansen added a comment - With the patched hdfs, we're still seeing slowdowns. For the 1-byte reads: 99.7% of requests have an average latency of 4.5ms 0.3% of requests have an average latency of 870 ms (selected for latency > 500ms) So we're spending 34% of our wall-clock time waiting for the slow reads. This is better than the unpatched 2.2.6, but still a bit rough. Running the same experiment against 2.2.0.0 gave 0 reads at more than 500ms.
        Hide
        bobhansen Bob Hansen added a comment -

        Testing the latest patch as part of a full Hadoop build (rather than just a set of patched jars over an older Hadoop build) shows much less variance. After a warm-up period, we had >500k short requests < 1000ms and 0 at >= 1000ms.

        Let's call this a tentative success while we continue testing.

        I've reviewed the code. We can probably drop the default nio thread count down from 100 threads to the number of CPUs at a maximum. Other than that, +1.

        Show
        bobhansen Bob Hansen added a comment - Testing the latest patch as part of a full Hadoop build (rather than just a set of patched jars over an older Hadoop build) shows much less variance. After a warm-up period, we had >500k short requests < 1000ms and 0 at >= 1000ms. Let's call this a tentative success while we continue testing. I've reviewed the code. We can probably drop the default nio thread count down from 100 threads to the number of CPUs at a maximum. Other than that, +1.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Thanks Bob Hansen for review! Patch V4 fixed the default number of threads and checkstyle issues. QA report is too old to show test failures. Let's see next one.

        Show
        xiaobingo Xiaobing Zhou added a comment - Thanks Bob Hansen for review! Patch V4 fixed the default number of threads and checkstyle issues. QA report is too old to show test failures. Let's see next one.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 19m 40s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 5s There were no new javac warning messages.
        +1 javadoc 10m 13s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 6s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 22m 53s Tests failed in hadoop-common.
        -1 hdfs tests 93m 23s Tests failed in hadoop-hdfs.
            163m 19s  



        Reason Tests
        Failed unit tests hadoop.ipc.TestRPC
        Timed out tests org.apache.hadoop.hdfs.server.mover.TestStorageMover
          org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761207/HDFS-8696.004.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 94dec5a
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12553/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12553/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12553/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12553/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 40s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 5s There were no new javac warning messages. +1 javadoc 10m 13s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 6s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 53s Tests failed in hadoop-common. -1 hdfs tests 93m 23s Tests failed in hadoop-hdfs.     163m 19s   Reason Tests Failed unit tests hadoop.ipc.TestRPC Timed out tests org.apache.hadoop.hdfs.server.mover.TestStorageMover   org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761207/HDFS-8696.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 94dec5a hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12553/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12553/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12553/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12553/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        Thanks everybody.

        I talked to a bunch of folks offline. Here is the recap:

        • After HDFS-7279, the DNs are capable of handling much more requests concurrently compared to the days that DN served WebHDFS traffic using Jetty.
        • The requests have high variances of latency as observed in this jira.
        • After poking at multiple parameters in Netty, it looks like that tweaking low / high water marks from (32k / 64k) to (256k / 1M) has resolved the issues.

        It looks to me that the results of experiments imply that there are high variances of latency in DFSInputStream#read(), so having a larger buffer by setting water marks definitely helps. On the other hand, having a higher water mark also means significant more buffer consumptions when there are many (1000+) concurrent connections. I don't think these tweaks should not be turned on by default. They should only be tuned for specific use cases.

        For this patch I would suggest (1) making only high / low water mark configurable. (2) leave low / high water marks as the defaults if when unspecified.

        Show
        wheat9 Haohui Mai added a comment - Thanks everybody. I talked to a bunch of folks offline. Here is the recap: After HDFS-7279 , the DNs are capable of handling much more requests concurrently compared to the days that DN served WebHDFS traffic using Jetty. The requests have high variances of latency as observed in this jira. After poking at multiple parameters in Netty, it looks like that tweaking low / high water marks from (32k / 64k) to (256k / 1M) has resolved the issues. It looks to me that the results of experiments imply that there are high variances of latency in DFSInputStream#read() , so having a larger buffer by setting water marks definitely helps. On the other hand, having a higher water mark also means significant more buffer consumptions when there are many (1000+) concurrent connections. I don't think these tweaks should not be turned on by default. They should only be tuned for specific use cases. For this patch I would suggest (1) making only high / low water mark configurable. (2) leave low / high water marks as the defaults if when unspecified.
        Hide
        bobhansen Bob Hansen added a comment -

        That seems a good, low-risk approach to the issue.

        I took the liberty of trimming the patch to just those changes.

        Show
        bobhansen Bob Hansen added a comment - That seems a good, low-risk approach to the issue. I took the liberty of trimming the patch to just those changes.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 38s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 55s There were no new javac warning messages.
        +1 javadoc 10m 7s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 21s The applied patch generated 4 new checkstyle issues (total was 405, now 409).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 12s Pre-build of native portion
        -1 hdfs tests 163m 32s Tests failed in hadoop-hdfs.
            208m 42s  



        Reason Tests
        Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2
          hadoop.hdfs.TestSafeMode



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761687/HDFS-8696.005.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / cc2b473
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12603/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12603/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12603/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12603/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 55s There were no new javac warning messages. +1 javadoc 10m 7s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 4 new checkstyle issues (total was 405, now 409). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 12s Pre-build of native portion -1 hdfs tests 163m 32s Tests failed in hadoop-hdfs.     208m 42s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.hdfs.TestSafeMode Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761687/HDFS-8696.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / cc2b473 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12603/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12603/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12603/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12603/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        The patch looks pretty good to me.

        Hi Bob Hansen, can you please clean up the checkstyle errors? +1 after addressing them.

        Show
        wheat9 Haohui Mai added a comment - The patch looks pretty good to me. Hi Bob Hansen , can you please clean up the checkstyle errors? +1 after addressing them.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Made patch V6 to fix check-style issues. Thanks Haohui Mai for review!

        Show
        xiaobingo Xiaobing Zhou added a comment - Made patch V6 to fix check-style issues. Thanks Haohui Mai for review!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 12s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 53s There were no new javac warning messages.
        +1 javadoc 10m 14s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 22s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 14s Pre-build of native portion
        -1 hdfs tests 160m 43s Tests failed in hadoop-hdfs.
            206m 49s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.mover.TestStorageMover
        Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761981/HDFS-8696.006.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1f707ec
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12640/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12640/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12640/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 53s There were no new javac warning messages. +1 javadoc 10m 14s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 160m 43s Tests failed in hadoop-hdfs.     206m 49s   Reason Tests Failed unit tests hadoop.hdfs.server.mover.TestStorageMover Timed out tests org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761981/HDFS-8696.006.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1f707ec hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12640/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12640/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12640/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -
        +  public static final String  DFS_WEBHDFS_CHANNEL_BUF_LOW_WATERMARK =
        +      "dfs.webhdfs.channel.buf.low.watermark";
        +  public static final int  DFS_WEBHDFS_CHANNEL_BUF_LOW_WATERMARK_DEFAULT =
        +      65535;
        +  public static final String  DFS_WEBHDFS_CHANNEL_BUF_HIGH_WATERMARK =
        +      "dfs.webhdfs.channel.buf.high.watermark";
        +  public static final int  DFS_WEBHDFS_CHANNEL_BUF_HIGH_WATERMARK_DEFAULT =
        +      131070;
        +
        

        Please reset the value to Netty's default. And it might make more sense to rename the configuration to dfs.webhdfs.netty.high/low.watermark to reflect the facts that they are implementation-specific.

        Show
        wheat9 Haohui Mai added a comment - + public static final String DFS_WEBHDFS_CHANNEL_BUF_LOW_WATERMARK = + "dfs.webhdfs.channel.buf.low.watermark" ; + public static final int DFS_WEBHDFS_CHANNEL_BUF_LOW_WATERMARK_DEFAULT = + 65535; + public static final String DFS_WEBHDFS_CHANNEL_BUF_HIGH_WATERMARK = + "dfs.webhdfs.channel.buf.high.watermark" ; + public static final int DFS_WEBHDFS_CHANNEL_BUF_HIGH_WATERMARK_DEFAULT = + 131070; + Please reset the value to Netty's default. And it might make more sense to rename the configuration to dfs.webhdfs.netty.high/low.watermark to reflect the facts that they are implementation-specific.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Posted patch 007 with correction! Thanks Haohui Mai!

        Show
        xiaobingo Xiaobing Zhou added a comment - Posted patch 007 with correction! Thanks Haohui Mai !
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 9s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 5s There were no new javac warning messages.
        +1 javadoc 10m 24s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 23s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 29s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 11s Pre-build of native portion
        -1 hdfs tests 0m 21s Tests failed in hadoop-hdfs.
            46m 34s  



        Reason Tests
        Failed build hadoop-hdfs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762052/HDFS-8696.007.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 06d1c90
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12653/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12653/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12653/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 5s There were no new javac warning messages. +1 javadoc 10m 24s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 23s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 0m 21s Tests failed in hadoop-hdfs.     46m 34s   Reason Tests Failed build hadoop-hdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762052/HDFS-8696.007.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 06d1c90 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12653/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12653/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12653/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -
        +
        +      this.httpServer.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK,
        +          conf.getInt(DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_LOW_WATERMARK,
        +              DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_LOW_WATERMARK_DEFAULT));
        +      this.httpServer.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK,
        +          conf.getInt(DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_HIGH_WATERMARK,
        +              DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_HIGH_WATERMARK_DEFAULT));
        +
               if (externalHttpChannel == null) {
        

        I assume that there are copy and paste errors here.

        Show
        wheat9 Haohui Mai added a comment - + + this .httpServer.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, + conf.getInt(DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_LOW_WATERMARK, + DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_LOW_WATERMARK_DEFAULT)); + this .httpServer.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, + conf.getInt(DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_HIGH_WATERMARK, + DFSConfigKeys.DFS_WEBHDFS_NETTY_CHANNEL_HIGH_WATERMARK_DEFAULT)); + if (externalHttpChannel == null ) { I assume that there are copy and paste errors here.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 4s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 49s There were no new javac warning messages.
        +1 javadoc 10m 2s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 22s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 9s Pre-build of native portion
        -1 hdfs tests 163m 21s Tests failed in hadoop-hdfs.
            208m 44s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12762196/HDFS-8696.008.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 71a81b6
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12665/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12665/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12665/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 49s There were no new javac warning messages. +1 javadoc 10m 2s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 9s Pre-build of native portion -1 hdfs tests 163m 21s Tests failed in hadoop-hdfs.     208m 44s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12762196/HDFS-8696.008.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 71a81b6 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12665/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12665/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12665/console This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Test failure is irrelevant. It passed in local setup with the patch.

        Show
        xiaobingo Xiaobing Zhou added a comment - Test failure is irrelevant. It passed in local setup with the patch.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 5s There were no new javac warning messages.
        +1 javadoc 10m 4s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 25s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse.
        +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 16s Pre-build of native portion
        -1 hdfs tests 168m 26s Tests failed in hadoop-hdfs.
            214m 50s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.security.TestDelegationToken
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.security.TestDelegationTokenForProxyUser



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764070/HDFS-8696.009.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 4c9497c
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12718/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12718/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12718/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12718/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 5s There were no new javac warning messages. +1 javadoc 10m 4s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 39s The patch built with eclipse:eclipse. +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 168m 26s Tests failed in hadoop-hdfs.     214m 50s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.security.TestDelegationToken   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.security.TestDelegationTokenForProxyUser Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764070/HDFS-8696.009.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4c9497c Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12718/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12718/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12718/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12718/console This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Checked the test failures are not related.

        Show
        xiaobingo Xiaobing Zhou added a comment - Checked the test failures are not related.
        Hide
        wheat9 Haohui Mai added a comment -

        The patch no longer applies to trunk anymore. Can you please rebase?

        Show
        wheat9 Haohui Mai added a comment - The patch no longer applies to trunk anymore. Can you please rebase?
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        New patch HDFS-8696.010.patch uploaded, thanks!

        Show
        xiaobingo Xiaobing Zhou added a comment - New patch HDFS-8696 .010.patch uploaded, thanks!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 19m 39s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 48s There were no new javac warning messages.
        +1 javadoc 10m 5s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 24s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 28s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 16s Pre-build of native portion
        -1 hdfs tests 163m 47s Tests failed in hadoop-hdfs.
            211m 2s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestAuditLogs



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764283/HDFS-8696.010.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 80d33b5
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12742/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12742/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12742/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12742/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 19m 39s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 10m 5s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 24s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 163m 47s Tests failed in hadoop-hdfs.     211m 2s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestAuditLogs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764283/HDFS-8696.010.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 80d33b5 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12742/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12742/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12742/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12742/console This message was automatically generated.
        Hide
        xiaobingo Xiaobing Zhou added a comment -

        Checked the test failures are not related.

        Show
        xiaobingo Xiaobing Zhou added a comment - Checked the test failures are not related.
        Hide
        wheat9 Haohui Mai added a comment -

        +1

        Show
        wheat9 Haohui Mai added a comment - +1
        Hide
        wheat9 Haohui Mai added a comment -

        I've committed the patch to trunk and branch-2. Thanks Xiaobing Zhou for the contribution.

        Show
        wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Xiaobing Zhou for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8543 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8543/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8543 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8543/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #469 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/469/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #469 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/469/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #461 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/461/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #461 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/461/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #436 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/436/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #436 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/436/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2376 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2376/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2376 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2376/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2404 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2404/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2404 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2404/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1200 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1200/)
        HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1200 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1200/ ) HDFS-8696 . Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. (wheat9: rev 6f335e4f0ea857ef11ff24c392bf6e549b5db406) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            xiaobingo Xiaobing Zhou
            Reporter:
            xiaobingo Xiaobing Zhou
          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development