Hadoop Common
  1. Hadoop Common
  2. HADOOP-8749

HADOOP-8031 changed the way in which relative xincludes are handled in Configuration.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.2-alpha
    • Component/s: conf
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The patch from HADOOP-8031 changed the xml parsing to use DocumentBuilder#parse(InputStream uri.openStream()) instead of DocumentBuilder#parse(String uri.toString()).I looked into the implementation of javax.xml.parsers.DocumentBuilder and org.xml.sax.InputSource and there is a difference when the DocumentBuilder parse(String) method is used versus parse(InputStream).

      1. HADOOP-8749.patch
        2 kB
        Ahmed Radwan
      2. HADOOP-8749_rev4.patch
        4 kB
        Ahmed Radwan
      3. HADOOP-8749_rev3.patch
        4 kB
        Ahmed Radwan
      4. HADOOP-8749_rev2.patch
        4 kB
        Ahmed Radwan

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/)
          HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/ ) HADOOP-8749 . HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/)
          HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/ ) HADOOP-8749 . HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2719 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2719/)
          HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2719 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2719/ ) HADOOP-8749 . HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2695 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2695/)
          HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2695 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2695/ ) HADOOP-8749 . HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2758 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2758/)
          HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2758 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2758/ ) HADOOP-8749 . HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. (ahmed via tucu) (Revision 1381703) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381703 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Ahmed. Committed to trunk and branch-2.

          Show
          Alejandro Abdelnur added a comment - Thanks Ahmed. Committed to trunk and branch-2.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543828/HADOOP-8749_rev4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1403//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1403//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543828/HADOOP-8749_rev4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1403//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1403//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543827/HADOOP-8749_rev3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1402//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1402//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543827/HADOOP-8749_rev3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1402//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1402//console This message is automatically generated.
          Hide
          Ahmed Radwan added a comment -

          I have modified the test case again to have a proper way of cleaning up the created directory.

          Show
          Ahmed Radwan added a comment - I have modified the test case again to have a proper way of cleaning up the created directory.
          Hide
          Ahmed Radwan added a comment -

          I missed adding the tmp directory creation to the previous patch (I had it locally and this is why it was successfully running). Here is the updated patch. The TestZKFailoverController failures seems unrelated (see HADOOP-8591).

          Show
          Ahmed Radwan added a comment - I missed adding the tmp directory creation to the previous patch (I had it locally and this is why it was successfully running). Here is the updated patch. The TestZKFailoverController failures seems unrelated (see HADOOP-8591 ).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543777/HADOOP-8749_rev2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.conf.TestConfiguration

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1400//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1400//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543777/HADOOP-8749_rev2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.conf.TestConfiguration +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1400//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1400//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          +1, looks good to me, please submit patch so that Hudson can check it out.

          Show
          Todd Lipcon added a comment - +1, looks good to me, please submit patch so that Hudson can check it out.
          Hide
          Ahmed Radwan added a comment -

          Thanks Todd, ATM and Steve!
          Here is the updated patch. I have added a test case to cover when a relative path is used in the XInclude.

          Show
          Ahmed Radwan added a comment - Thanks Todd, ATM and Steve! Here is the updated patch. I have added a test case to cover when a relative path is used in the XInclude.
          Hide
          Steve Loughran added a comment -

          any proposed patch ought to handle the case where the XML parser doesn't support XInclude; that's broken things before

          Show
          Steve Loughran added a comment - any proposed patch ought to handle the case where the XML parser doesn't support XInclude; that's broken things before
          Hide
          Aaron T. Myers added a comment -

          Great investigation, Ahmed. I noticed that this behavior had changed, but from a quick look couldn't find any obviously relevant changes.

          Show
          Aaron T. Myers added a comment - Great investigation, Ahmed. I noticed that this behavior had changed, but from a quick look couldn't find any obviously relevant changes.
          Hide
          Todd Lipcon added a comment -

          I verified that this fix worked on my cluster. Will wait for the test case, though. Thanks for looking into this, Ahmed!

          Show
          Todd Lipcon added a comment - I verified that this fix worked on my cluster. Will wait for the test case, though. Thanks for looking into this, Ahmed!
          Hide
          Ahmed Radwan added a comment -

          I'll be adding a test case.

          Show
          Ahmed Radwan added a comment - I'll be adding a test case.
          Hide
          Ahmed Radwan added a comment -

          Basically we need to use parse(InputStream is, String systemId) which provides a base for resolving relative URIs. Here is a new patch that fixes this issue.

          Show
          Ahmed Radwan added a comment - Basically we need to use parse(InputStream is, String systemId) which provides a base for resolving relative URIs. Here is a new patch that fixes this issue.

            People

            • Assignee:
              Ahmed Radwan
              Reporter:
              Ahmed Radwan
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development