Hadoop Common
  1. Hadoop Common
  2. HADOOP-8525

Provide Improved Traceability for Configuration

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None

      Description

      Configuration provides basic traceability to see where a config setting came from, but once the configuration is written out that information is written to a comment in the XML and then lost the next time the configuration is read back in. It would really be great to be able to store a complete history of where the config came from in the XML, so that it can then be retrieved later for debugging.

      1. HADOOP-8525.txt
        27 kB
        Robert Joseph Evans
      2. HADOOP-8525.txt
        26 kB
        Robert Joseph Evans
      3. HADOOP-8525.txt
        12 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          8d 2h 40m 2 Robert Joseph Evans 06/Jul/12 22:31
          Open Open Patch Available Patch Available
          6d 22h 10m 3 Robert Joseph Evans 06/Jul/12 22:31
          Patch Available Patch Available Resolved Resolved
          3d 19h 36m 1 Robert Joseph Evans 10/Jul/12 18:08
          Resolved Resolved Closed Closed
          93d 36m 1 Arun C Murthy 11/Oct/12 18:45
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-4375 [ MAPREDUCE-4375 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-4375 [ MAPREDUCE-4375 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322473 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Fix Version/s 2.1.0-alpha [ 12321441 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1133 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1133 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1133/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #310 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/310/)
          svn merge -r 1359774:1359777 FIXES: HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359789)

          Result = UNSTABLE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359789
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #310 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/310/ ) svn merge -r 1359774:1359777 FIXES: HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359789) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359789 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2456 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2456/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2456 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2456/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2506 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2506/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2506 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2506/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2438 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2438/)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359777)
          HADOOP-8525. Provide Improved Traceability for Configuration (bobby) (Revision 1359775)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2438 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2438/ ) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359777) HADOOP-8525 . Provide Improved Traceability for Configuration (bobby) (Revision 1359775) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359777 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359775 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfServlet.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 2.0.1-alpha [ 12321441 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Resolution Fixed [ 1 ]
          Hide
          Harsh J added a comment -

          +1, the patch looks good. Just applied this and tried it out, and the results look good too

          Show
          Harsh J added a comment - +1, the patch looks good. Just applied this and tried it out, and the results look good too
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12535438/HADOOP-8525.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1179//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1179//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535438/HADOOP-8525.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1179//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1179//console This message is automatically generated.
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment HADOOP-8525.txt [ 12535438 ]
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Robert Joseph Evans added a comment -

          Canecling patch to address test failures

          Show
          Robert Joseph Evans added a comment - Canecling patch to address test failures
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12533874/HADOOP-8525.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.io.file.tfile.TestTFileByteArrays
          org.apache.hadoop.conf.TestConfServlet
          org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1149//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1149//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533874/HADOOP-8525.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.io.file.tfile.TestTFileByteArrays org.apache.hadoop.conf.TestConfServlet org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1149//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1149//console This message is automatically generated.
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment HADOOP-8525.txt [ 12533874 ]
          Hide
          Robert Joseph Evans added a comment -

          I have updated the code so that the Writeable part maintains the sources. The source can be provided when setting a config, or when adding in a stream resource. I also updated the GenericOptionsParser to also set the source as command line for those configs that are set on the command line.

          Show
          Robert Joseph Evans added a comment - I have updated the code so that the Writeable part maintains the sources. The source can be provided when setting a config, or when adding in a stream resource. I also updated the GenericOptionsParser to also set the source as command line for those configs that are set on the command line.
          Hide
          Robert Joseph Evans added a comment -

          Yes that would make since to update the Writable methods. I remember looking and they are not really used, but they are still there.

          Show
          Robert Joseph Evans added a comment - Yes that would make since to update the Writable methods. I remember looking and they are not really used, but they are still there.
          Hide
          Harsh J added a comment -

          Overall this looks pretty good!

          One comment regarding completeness: I noticed Configuration being a Writable. I've not really seen a use of Configuration being used as Writable, but perhaps the readFields and write methods need to be updated too, without breaking compatibility? Or we may document that serialization via Writable will not include sources of configs.

          Other than this, +1.

          Show
          Harsh J added a comment - Overall this looks pretty good! One comment regarding completeness: I noticed Configuration being a Writable. I've not really seen a use of Configuration being used as Writable, but perhaps the readFields and write methods need to be updated too, without breaking compatibility? Or we may document that serialization via Writable will not include sources of configs. Other than this, +1.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12533538/HADOOP-8525.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.viewfs.TestViewFsTrash
          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.io.file.tfile.TestTFileByteArrays
          org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1142//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1142//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533538/HADOOP-8525.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.viewfs.TestViewFsTrash org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.io.file.tfile.TestTFileByteArrays org.apache.hadoop.io.file.tfile.TestTFileJClassComparatorByteArrays +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1142//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1142//console This message is automatically generated.
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Robert Joseph Evans added a comment -

          Canceling patch after trying to use the results as part of MAPREDUCE-4375. It is working as designed, but with the history server we are getting a lot of

          org.apache.hadoop.hdfs.DFSClient$DFSDataInputStream@129b073

          so we need to add in a way to include a name along with the InputStream when adding in a resource.

          Show
          Robert Joseph Evans added a comment - Canceling patch after trying to use the results as part of MAPREDUCE-4375 . It is working as designed, but with the history server we are getting a lot of org.apache.hadoop.hdfs.DFSClient$DFSDataInputStream@129b073 so we need to add in a way to include a name along with the InputStream when adding in a resource.
          Robert Joseph Evans made changes -
          Link This issue blocks MAPREDUCE-4375 [ MAPREDUCE-4375 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Robert Joseph Evans made changes -
          Attachment HADOOP-8525.txt [ 12533538 ]
          Hide
          Robert Joseph Evans added a comment -

          This adds in the traceability. I have tested this on a small cluster.

          Show
          Robert Joseph Evans added a comment - This adds in the traceability. I have tested this on a small cluster.
          Harsh J made changes -
          Link This issue supercedes HADOOP-8524 [ HADOOP-8524 ]
          Hide
          Robert Joseph Evans added a comment -

          I think the only big change here is a small addition to the XML format, so that we can add in a list of previous files it was read from and remove the comment that more or less gives the same information. Internally the Map storing this into will have to now point to a list instead of a single value. It looks fairly straight forward.

          Show
          Robert Joseph Evans added a comment - I think the only big change here is a small addition to the XML format, so that we can add in a list of previous files it was read from and remove the comment that more or less gives the same information. Internally the Map storing this into will have to now point to a list instead of a single value. It looks fairly straight forward.
          Robert Joseph Evans made changes -
          Field Original Value New Value
          Assignee Robert Joseph Evans [ revans2 ]
          Robert Joseph Evans created issue -

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development