Hadoop Common
  1. Hadoop Common
  2. HADOOP-7436

Bundle Log4j socket appender Metrics plugin in Hadoop

    Details

    • Type: New Feature New Feature
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      Java 6

    • Release Note:
      New SocketSink plugin for Metrics2 framework to stream metrics to Chukwa SocketAdaptor

      Description

      Modify the hadoop-metrics.properties to use org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext for emitting metrics. It is preferred to reduce the number of manual steps that is required to enable other monitored hadoop cluster by moving the plugin code into hadoop code base. It is similar to bundling Ganglia metrics plugin in Hadoop code base.

      1. HADOOP-7436.patch
        13 kB
        Eric Yang
      2. HADOOP-7436-1.patch
        13 kB
        Eric Yang
      3. HADOOP-7436-2.patch
        10 kB
        Sandhya
      4. HADOOP-7436-3.patch
        11 kB
        Sandhya
      5. HADOOP-7436-4-branch-2.patch
        13 kB
        Eric Yang
      6. HADOOP-7436-4-trunk.patch
        13 kB
        Eric Yang
      7. HADOOP-7436-5-branch2.patch
        13 kB
        Eric Yang
      8. HADOOP-7436-5-trunk.patch
        13 kB
        Eric Yang
      9. HADOOP-7436-6-trunk.patch
        13 kB
        Eric Yang
      10. HADOOP-7436-7-trunk.patch
        13 kB
        Eric Yang
      11. HADOOP-7436-8-trunk.patch
        13 kB
        Eric Yang

        Activity

        Eric Yang created issue -
        Hide
        Todd Lipcon added a comment -

        I disagree – I think what we need is a better plugin system. Contrib like this tends to aggregate broken code that no one wants to maintain (witness hdfsproxy tests that have been failing >1 year)

        Show
        Todd Lipcon added a comment - I disagree – I think what we need is a better plugin system. Contrib like this tends to aggregate broken code that no one wants to maintain (witness hdfsproxy tests that have been failing >1 year)
        Hide
        Eric Yang added a comment -

        Metrics plugin is like a driver software. Linux kernel bundles compatible drivers in the source tarball. This is no different, it is easier to certify a software driver if it is included in the kernel source with test cases. If a broken driver is not fixed, it is easy to remove. The power and responsibilities are in the hand of the Hadoop PMC. If Hadoop community don't make integration easy, then it depends on third party to make the integration happen. The distributed integration approach only drains developers from Hadoop community. This is something I would like to avoid.

        Show
        Eric Yang added a comment - Metrics plugin is like a driver software. Linux kernel bundles compatible drivers in the source tarball. This is no different, it is easier to certify a software driver if it is included in the kernel source with test cases. If a broken driver is not fixed, it is easy to remove. The power and responsibilities are in the hand of the Hadoop PMC. If Hadoop community don't make integration easy, then it depends on third party to make the integration happen. The distributed integration approach only drains developers from Hadoop community. This is something I would like to avoid.
        Hide
        Todd Lipcon added a comment -

        The difference is that Linux has a sane maintainer system, where each subtree has someone who is committed to its maintenance. We don't have such a thing.

        In theory, it's easy to remove broken code, but in practice, when we tried to do this with hdfxproxy, people -1ed its removal and then didn't step up to fix the tests (which are still failing).

        Show
        Todd Lipcon added a comment - The difference is that Linux has a sane maintainer system, where each subtree has someone who is committed to its maintenance. We don't have such a thing. In theory, it's easy to remove broken code, but in practice, when we tried to do this with hdfxproxy, people -1ed its removal and then didn't step up to fix the tests (which are still failing).
        Hide
        Eric Yang added a comment -

        Chukwa uses a simple mechanism, serialize metrics to JSON string, and stream JSON string to external system using Log4J socket appender. Chukwa metrics plugin is in fact generic. Hadoop currently has FileSink plugin to demo how to interface with Metrics 2 framework. It would be useful to have a generic Metrics 2 plugin that can stream metrics to any monitoring system that implements Log4J socket appender server. Hence, this should not generate unmaintained legacy code but a good interface for external monitoring systems. If I rename this plugin as Log4J Socket Metrics plugin, would this be a reasonable plugin to have in Hadoop?

        Show
        Eric Yang added a comment - Chukwa uses a simple mechanism, serialize metrics to JSON string, and stream JSON string to external system using Log4J socket appender. Chukwa metrics plugin is in fact generic. Hadoop currently has FileSink plugin to demo how to interface with Metrics 2 framework. It would be useful to have a generic Metrics 2 plugin that can stream metrics to any monitoring system that implements Log4J socket appender server. Hence, this should not generate unmaintained legacy code but a good interface for external monitoring systems. If I rename this plugin as Log4J Socket Metrics plugin, would this be a reasonable plugin to have in Hadoop?
        Hide
        Todd Lipcon added a comment -

        Yea, if it's that generic, I suppose it makes sense. I had figured it was a Chukwa-specific protocol and would have to pull in chukwa as a dependency.

        Show
        Todd Lipcon added a comment - Yea, if it's that generic, I suppose it makes sense. I had figured it was a Chukwa-specific protocol and would have to pull in chukwa as a dependency.
        Hide
        Luke Lu added a comment -

        Yes, this is as reasonable as bundling Ganglia sinks, IMO.

        Show
        Luke Lu added a comment - Yes, this is as reasonable as bundling Ganglia sinks, IMO.
        Hide
        Todd Lipcon added a comment -

        If not for history, I'd have argued against the Ganglia sinks, too, fwiw Though worth noting that Ganglia is vastly more commonly used than Chukwa.

        Show
        Todd Lipcon added a comment - If not for history, I'd have argued against the Ganglia sinks, too, fwiw Though worth noting that Ganglia is vastly more commonly used than Chukwa.
        Hide
        Eric Yang added a comment -

        Added SocketSink for Metrics2 to stream metrics to Chukwa SocketAdaptor or generic Log4JSocketAppender server.

        Show
        Eric Yang added a comment - Added SocketSink for Metrics2 to stream metrics to Chukwa SocketAdaptor or generic Log4JSocketAppender server.
        Eric Yang made changes -
        Field Original Value New Value
        Attachment HADOOP-7436.patch [ 12485321 ]
        Eric Yang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note New SocketSink plugin for Metrics2 framework to stream metrics to Chukwa SocketAdaptor
        Affects Version/s 0.23.0 [ 12315569 ]
        Fix Version/s 0.23.0 [ 12315569 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12485321/HADOOP-7436.patch
        against trunk revision 1143149.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12485321/HADOOP-7436.patch against trunk revision 1143149. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/694//console This message is automatically generated.
        Hide
        Eric Yang added a comment -

        Fixed findbugs warning.

        Show
        Eric Yang added a comment - Fixed findbugs warning.
        Eric Yang made changes -
        Attachment HADOOP-7436-1.patch [ 12485339 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12485339/HADOOP-7436-1.patch
        against trunk revision 1143149.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 5 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12485339/HADOOP-7436-1.patch against trunk revision 1143149. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/695//console This message is automatically generated.
        Arun C Murthy made changes -
        Fix Version/s 0.24.0 [ 12317652 ]
        Fix Version/s 0.23.0 [ 12315569 ]
        Eric Yang made changes -
        Summary Bundle Chukwa Metrics plugin in Hadoop Bundle Log4j socket appender Metrics plugin in Hadoop
        Description For monitoring hadoop cluster with Chukwa, the current step is to copy chukwa-hadoop-*-client.jar and json-simple to hadoop classpath. (i.e. $HADOOP_HOME/lib or $HADOOP_PREFIX/share/hadoop/lib), and modify the hadoop-metrics.properties to use org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext for emitting metrics. It is preferred to reduce the number of manual steps that is required to enable chukwa monitored hadoop cluster by moving the plugin code into hadoop code base. It is similar to bundling Ganglia metrics plugin in Hadoop code base. Modify the hadoop-metrics.properties to use org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext for emitting metrics. It is preferred to reduce the number of manual steps that is required to enable other monitored hadoop cluster by moving the plugin code into hadoop code base. It is similar to bundling Ganglia metrics plugin in Hadoop code base.
        Hide
        Luke Lu added a comment -

        IMO, We should avoid referencing chukwa in these patches, as they're just generic SocketSinks to send metrics as JSON, even though they're originally from chukwa.

        Show
        Luke Lu added a comment - IMO, We should avoid referencing chukwa in these patches, as they're just generic SocketSinks to send metrics as JSON, even though they're originally from chukwa.
        Harsh J made changes -
        Fix Version/s 0.24.0 [ 12317652 ]
        Hide
        Harsh J added a comment -

        Hey Eric - Mind addressing Luke's comments above with a new patch please?

        Show
        Harsh J added a comment - Hey Eric - Mind addressing Luke's comments above with a new patch please?
        Hide
        Sandhya added a comment -

        This is a generic socket adaptor that emits metrics in json format.

        Show
        Sandhya added a comment - This is a generic socket adaptor that emits metrics in json format.
        Sandhya made changes -
        Attachment HADOOP-7436-2.patch [ 12673629 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12673629/HADOOP-7436-2.patch
        against trunk revision 1efd9c9.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        -1 release audit. The applied patch generated 2 release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.ha.TestZKFailoverControllerStress

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//artifact/patchprocess/patchReleaseAuditProblems.txt
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12673629/HADOOP-7436-2.patch against trunk revision 1efd9c9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 2 release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverControllerStress +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4878//console This message is automatically generated.
        Hide
        Sandhya added a comment -

        From the stack trace of the failed testcase, the patch HADOOP-7436-2.patch is not causing that failure. I would have to include the missing Apache license header in both files in the patch.
        Thanks
        Sandhya

        Show
        Sandhya added a comment - From the stack trace of the failed testcase, the patch HADOOP-7436 -2.patch is not causing that failure. I would have to include the missing Apache license header in both files in the patch. Thanks Sandhya
        Hide
        Sandhya added a comment -

        Added apache license header .

        Show
        Sandhya added a comment - Added apache license header .
        Sandhya made changes -
        Attachment HADOOP-7436-3.patch [ 12673680 ]
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12673680/HADOOP-7436-3.patch
        against trunk revision e16e25a.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4881//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4881//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12673680/HADOOP-7436-3.patch against trunk revision e16e25a. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4881//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4881//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 30s There were no new javac warning messages.
        +1 javadoc 9m 38s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 4s The applied patch generated 68 new checkstyle issues (total was 0, now 68).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 22m 40s Tests passed in hadoop-common.
            59m 40s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12673680/HADOOP-7436-3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / e8d0ee5
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 30s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 4s The applied patch generated 68 new checkstyle issues (total was 0, now 68). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 1m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 22m 40s Tests passed in hadoop-common.     59m 40s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12673680/HADOOP-7436-3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e8d0ee5 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6448/console This message was automatically generated.
        Allen Wittenauer made changes -
        Labels BB2015-05-TBR
        Hide
        Eric Yang added a comment -

        Updated file structure to match current trunk.

        Show
        Eric Yang added a comment - Updated file structure to match current trunk.
        Eric Yang made changes -
        Attachment HADOOP-7436-4-trunk.patch [ 12740030 ]
        Hide
        Eric Yang added a comment -

        Same patch for branch 2.

        Show
        Eric Yang added a comment - Same patch for branch 2.
        Eric Yang made changes -
        Attachment HADOOP-7436-4-branch-2.patch [ 12740031 ]
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 58s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 50s There were no new javac warning messages.
        +1 javadoc 9m 56s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 4s The applied patch generated 22 new checkstyle issues (total was 0, now 22).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 37s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 23m 7s Tests passed in hadoop-common.
            63m 24s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12740031/HADOOP-7436-4-branch-2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 5dbc8c9
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 58s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 50s There were no new javac warning messages. +1 javadoc 9m 56s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 4s The applied patch generated 22 new checkstyle issues (total was 0, now 22). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 23m 7s Tests passed in hadoop-common.     63m 24s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740031/HADOOP-7436-4-branch-2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 5dbc8c9 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6982/console This message was automatically generated.
        Hide
        Eric Yang added a comment -

        Fixed check style issues.

        Show
        Eric Yang added a comment - Fixed check style issues.
        Eric Yang made changes -
        Attachment HADOOP-7436-5-trunk.patch [ 12740158 ]
        Attachment HADOOP-7436-5-branch2.patch [ 12740159 ]
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 0s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 40s There were no new javac warning messages.
        +1 javadoc 9m 51s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 4s The applied patch generated 21 new checkstyle issues (total was 0, now 21).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 22m 11s Tests passed in hadoop-common.
            62m 11s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12740159/HADOOP-7436-5-branch2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / cc43288
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/artifact/patchprocess/diffcheckstylehadoop-common.txt
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 0s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 9m 51s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 4s The applied patch generated 21 new checkstyle issues (total was 0, now 21). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 11s Tests passed in hadoop-common.     62m 11s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740159/HADOOP-7436-5-branch2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / cc43288 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/artifact/patchprocess/diffcheckstylehadoop-common.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6984/console This message was automatically generated.
        Hide
        Eric Yang added a comment -

        Fix the same problem again. Seems like my laptop is suffering from Samsung TRIM defect.

        Show
        Eric Yang added a comment - Fix the same problem again. Seems like my laptop is suffering from Samsung TRIM defect.
        Eric Yang made changes -
        Attachment HADOOP-7436-6-trunk.patch [ 12740424 ]
        Hide
        Eric Yang added a comment -

        Trigger patch for testing again.

        Show
        Eric Yang added a comment - Trigger patch for testing again.
        Eric Yang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Eric Yang added a comment -

        Rebase to current trunk.

        Show
        Eric Yang added a comment - Rebase to current trunk.
        Eric Yang made changes -
        Attachment HADOOP-7436-7-trunk.patch [ 12777630 ]
        Eric Yang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 4s HADOOP-7436 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777630/HADOOP-7436-7-trunk.patch
        JIRA Issue HADOOP-7436
        Powered by Apache Yetus 0.1.0 http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8239/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 4s HADOOP-7436 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777630/HADOOP-7436-7-trunk.patch JIRA Issue HADOOP-7436 Powered by Apache Yetus 0.1.0 http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8239/console This message was automatically generated.
        Eric Yang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Eric Yang added a comment -

        Rebase patch to trunk.

        Show
        Eric Yang added a comment - Rebase patch to trunk.
        Eric Yang made changes -
        Attachment HADOOP-7436-8-trunk.patch [ 12778384 ]
        Eric Yang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 11m 48s trunk passed
        +1 compile 14m 34s trunk passed with JDK v1.8.0_66
        +1 compile 13m 2s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 1m 32s trunk passed
        +1 mvneclipse 0m 22s trunk passed
        +1 findbugs 2m 47s trunk passed
        +1 javadoc 1m 28s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 35s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 27s the patch passed
        +1 compile 14m 14s the patch passed with JDK v1.8.0_66
        +1 javac 14m 14s the patch passed
        +1 compile 12m 44s the patch passed with JDK v1.7.0_91
        +1 javac 12m 44s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 1m 30s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 56s the patch passed
        +1 javadoc 1m 28s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 32s the patch passed with JDK v1.7.0_91
        -1 unit 10m 43s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 9m 49s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 35s Patch does not generate ASF License warnings.
        107m 58s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestFind
          hadoop.ha.TestZKFailoverController
          hadoop.test.TestTimedOutTestsListener
        JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.find.TestFind
          hadoop.security.ssl.TestReloadingX509TrustManager



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778384/HADOOP-7436-8-trunk.patch
        JIRA Issue HADOOP-7436
        Optional Tests asflicense mvnsite unit compile javac javadoc mvninstall findbugs checkstyle
        uname Linux 61284f5e0247 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6da9dee
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 11m 48s trunk passed +1 compile 14m 34s trunk passed with JDK v1.8.0_66 +1 compile 13m 2s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 32s trunk passed +1 mvneclipse 0m 22s trunk passed +1 findbugs 2m 47s trunk passed +1 javadoc 1m 28s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 35s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 27s the patch passed +1 compile 14m 14s the patch passed with JDK v1.8.0_66 +1 javac 14m 14s the patch passed +1 compile 12m 44s the patch passed with JDK v1.7.0_91 +1 javac 12m 44s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 22s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 56s the patch passed +1 javadoc 1m 28s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 32s the patch passed with JDK v1.7.0_91 -1 unit 10m 43s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 9m 49s hadoop-common in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 107m 58s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestFind   hadoop.ha.TestZKFailoverController   hadoop.test.TestTimedOutTestsListener JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.find.TestFind   hadoop.security.ssl.TestReloadingX509TrustManager Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778384/HADOOP-7436-8-trunk.patch JIRA Issue HADOOP-7436 Optional Tests asflicense mvnsite unit compile javac javadoc mvninstall findbugs checkstyle uname Linux 61284f5e0247 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6da9dee findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8269/console This message was automatically generated.
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        1626d 3h 35m 2 Eric Yang 18/Dec/15 00:06
        Open Open Patch Available Patch Available
        5d 48m 3 Eric Yang 18/Dec/15 00:06

          People

          • Assignee:
            Eric Yang
            Reporter:
            Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:

              Development