Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Issue Links

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #673 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/673/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #677 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/677/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #677 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/677/ )
        Hide
        Todd Lipcon added a comment -

        Committed to trunk. Thanks, Luke.

        Show
        Todd Lipcon added a comment - Committed to trunk. Thanks, Luke.
        Hide
        Todd Lipcon added a comment -

        cool, thanks Luke. +1 on this patch to fix the tests, then.

        Show
        Todd Lipcon added a comment - cool, thanks Luke. +1 on this patch to fix the tests, then.
        Hide
        Luke Lu added a comment -

        The only difference between the new behavior and metrics v1 is that in metrics v1, the metrics related mbeans are started whether or not metrics context are configured.

        In hindsight, I think I should've treated missing config as default/empty config for better compatibility and less surprises. I just opened HADOOP-7306 to revert the metrics system to the old behavior.

        Show
        Luke Lu added a comment - The only difference between the new behavior and metrics v1 is that in metrics v1, the metrics related mbeans are started whether or not metrics context are configured. In hindsight, I think I should've treated missing config as default/empty config for better compatibility and less surprises. I just opened HADOOP-7306 to revert the metrics system to the old behavior.
        Hide
        Todd Lipcon added a comment -

        It seems like it would be straight-forward to have a missing .properties file act like the default one that we check into conf/ (ie FileSink). That would make it less of an incompatible change, right?

        Show
        Todd Lipcon added a comment - It seems like it would be straight-forward to have a missing .properties file act like the default one that we check into conf/ (ie FileSink). That would make it less of an incompatible change, right?
        Hide
        Luke Lu added a comment -

        This seems like an incompatible change.

        Yes. It's an incompatible change and marked as such on related issues (HADOOP-6918, HDFS-1117).

        Show
        Luke Lu added a comment - This seems like an incompatible change. Yes. It's an incompatible change and marked as such on related issues ( HADOOP-6918 , HDFS-1117 ).
        Hide
        Luke Lu added a comment -

        If the properties file is missing, the metrics system would be initialized but not started (there is a warning about missing config file), which means the metrics related mbeans will not be registered, besides the metrics system control mbean, which can be used to (re)start the metrics system dynamically when the properties file is fixed/updated. Note the metrics itself is still accumulated and updated, just the publishing (periodic snapshotting and pushing to sinks and mbeans) part is stopped temporarily.

        This only affects admin/server (not normal users) configs which should be included in the distribution. See the compatibility discussions on the mailing lists (links on HADOOP-6728) for details.

        Show
        Luke Lu added a comment - If the properties file is missing, the metrics system would be initialized but not started (there is a warning about missing config file), which means the metrics related mbeans will not be registered, besides the metrics system control mbean, which can be used to (re)start the metrics system dynamically when the properties file is fixed/updated. Note the metrics itself is still accumulated and updated, just the publishing (periodic snapshotting and pushing to sinks and mbeans) part is stopped temporarily. This only affects admin/server (not normal users) configs which should be included in the distribution. See the compatibility discussions on the mailing lists (links on HADOOP-6728 ) for details.
        Hide
        Todd Lipcon added a comment -

        One question about this: does this mean that, if the hadoop-metrics2.properties file is missing from a user's conf/ dir, Hadoop won't function correctly?

        This seems like an incompatible change since a user's conf directory from a prior installation won't function on trunk.

        Am I mis-interpreting the problem and its fix?

        Show
        Todd Lipcon added a comment - One question about this: does this mean that, if the hadoop-metrics2.properties file is missing from a user's conf/ dir, Hadoop won't function correctly? This seems like an incompatible change since a user's conf directory from a prior installation won't function on trunk. Am I mis-interpreting the problem and its fix?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12479682/hdfs-1922-conf-v1.patch
        against trunk revision 1124459.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFileConcurrentReader

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479682/hdfs-1922-conf-v1.patch against trunk revision 1124459. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/572//console This message is automatically generated.
        Hide
        Luke Lu added a comment -

        Patch v1 to copy the metrics properties to test classpath. I thought the test classpath includes the normal classpath.

        Show
        Luke Lu added a comment - Patch v1 to copy the metrics properties to test classpath. I thought the test classpath includes the normal classpath.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        HDFS-1869. BTW, build #549 was started on May 18.

        Show
        Tsz Wo Nicholas Sze added a comment - HDFS-1869 . BTW, build #549 was started on May 18.
        Hide
        Luke Lu added a comment -

        HDFS-1117 was committed but TestJMXGet is still failing; see build #549.

        Can you tell me which JIRA that build is for? I recall the test were fine for a while after HDFS-1117 was committed.

        Show
        Luke Lu added a comment - HDFS-1117 was committed but TestJMXGet is still failing; see build #549. Can you tell me which JIRA that build is for? I recall the test were fine for a while after HDFS-1117 was committed.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        It simply fails by

        java.lang.NumberFormatException: For input string: ""
        	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
        	at java.lang.Long.parseLong(Long.java:431)
        	at java.lang.Long.parseLong(Long.java:468)
        	at org.apache.hadoop.tools.TestJMXGet.testNameNode(TestJMXGet.java:102)
        
        Show
        Tsz Wo Nicholas Sze added a comment - It simply fails by java.lang.NumberFormatException: For input string: "" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Long.parseLong(Long.java:431) at java.lang.Long.parseLong(Long.java:468) at org.apache.hadoop.tools.TestJMXGet.testNameNode(TestJMXGet.java:102)
        Hide
        Tsz Wo Nicholas Sze added a comment -

        HDFS-1117 was committed but TestJMXGet is still failing; see build #549.

        Show
        Tsz Wo Nicholas Sze added a comment - HDFS-1117 was committed but TestJMXGet is still failing; see build #549 .
        Hide
        Luke Lu added a comment -

        The issue should be resolved once HDFS-1117 is committed.

        Show
        Luke Lu added a comment - The issue should be resolved once HDFS-1117 is committed.
        Hide
        Todd Lipcon added a comment -

        Is someone working on this? The build has been broken for a week.

        Show
        Todd Lipcon added a comment - Is someone working on this? The build has been broken for a week.

          People

          • Assignee:
            Luke Lu
            Reporter:
            Matt Foley
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development