Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: build
    • Labels:
    • Environment:

      FreeBSD 8.2, 64bit, Artifactory

    • Target Version/s:
    • Tags:
      maven

      Description

      This is backport of HADOOP-8268 to 0.23 branch. It fixes invalid pom.xml which allows them to be uploaded into artifactory maven repository management and adds schema declarations which allows to use XML validating tools.

      1. hadoop-invalid-pom-023.txt
        37 kB
        Radim Kolar
      2. hadoop-invalid-pom-023-2.txt
        37 kB
        Radim Kolar
      3. hadoop-invalid-pom-023-3.txt
        1.0 kB
        Radim Kolar

        Activity

        Hide
        Robert Joseph Evans added a comment -

        Thanks Radim for the fix. I forgot to resolve this earlier. I put this into branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Radim for the fix. I forgot to resolve this earlier. I put this into branch-0.23
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #305 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/305/)
        HADOOP-8543. Invalid pom.xml files on 0.23 branch Updated to fix a bug in orriginal patch. (Revision 1357785)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1357785
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-project-dist/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #305 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/305/ ) HADOOP-8543 . Invalid pom.xml files on 0.23 branch Updated to fix a bug in orriginal patch. (Revision 1357785) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1357785 Files : /hadoop/common/branches/branch-0.23/hadoop-project-dist/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #304 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/304/)
        HADOOP-8543. Invalid pom.xml files on 0.23 branch (Radim Kolar via bobby) (Revision 1356941)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1356941
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-assemblies/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-annotations/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth-examples/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-dist/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-project-dist/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-archives/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-datajoin/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-gridmix/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-pipes/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-tools-dist/pom.xml
        • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
        • /hadoop/common/branches/branch-0.23/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #304 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/304/ ) HADOOP-8543 . Invalid pom.xml files on 0.23 branch (Radim Kolar via bobby) (Revision 1356941) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1356941 Files : /hadoop/common/branches/branch-0.23/hadoop-assemblies/pom.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/branches/branch-0.23/hadoop-common-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-dist/pom.xml /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-project-dist/pom.xml /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-archives/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-datajoin/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-extras/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-gridmix/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-rumen/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-streaming/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-tools-dist/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml /hadoop/common/branches/branch-0.23/pom.xml
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535003/hadoop-invalid-pom-023-3.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1169//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535003/hadoop-invalid-pom-023-3.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1169//console This message is automatically generated.
        Hide
        Radim Kolar added a comment -

        hadoop-invalid-pom-023-3.txt needs to be applied into current git tree. it fixes bug introduced by -023-2.txt patch

        Show
        Radim Kolar added a comment - hadoop-invalid-pom-023-3.txt needs to be applied into current git tree. it fixes bug introduced by -023-2.txt patch
        Hide
        Radim Kolar added a comment -

        hadoop-project-dist/pom.xml was patched incorrectly. i will make separate patch for this file.

        Show
        Radim Kolar added a comment - hadoop-project-dist/pom.xml was patched incorrectly. i will make separate patch for this file.
        Hide
        Robert Joseph Evans added a comment -

        Thanks for the fix Radim +1, I checked this into branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks for the fix Radim +1, I checked this into branch-0.23
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12534662/hadoop-invalid-pom-023-2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1165//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12534662/hadoop-invalid-pom-023-2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1165//console This message is automatically generated.
        Hide
        Radim Kolar added a comment -

        Tested with mvn compile openJDK7

        Show
        Radim Kolar added a comment - Tested with mvn compile openJDK7
        Hide
        Robert Joseph Evans added a comment -

        Oh I see now hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml

         $ diff --ignore-space-change hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml ../branch-2/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
        ...
        488c480
        <                       which cygpath &gt;> /dev/null
        ---
        >                       which cygpath 2&gt; /dev/null
        531c523
        <                       which cygpath &gt;> /dev/null
        ---
        >                       which cygpath 2&gt; /dev/null
        

        It looks like there was a mixup in the cygpath line "2>" got changed to ">>" but still with invalid XML.

        Show
        Robert Joseph Evans added a comment - Oh I see now hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml $ diff --ignore-space-change hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml ../branch-2/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml ... 488c480 < which cygpath &gt;> /dev/null --- > which cygpath 2&gt; /dev/null 531c523 < which cygpath &gt;> /dev/null --- > which cygpath 2&gt; /dev/null It looks like there was a mixup in the cygpath line "2>" got changed to ">>" but still with invalid XML.
        Hide
        Robert Joseph Evans added a comment -

        branch-0.23 does not seem to build for me after applying the patch. I am using RHEL 5, mvn 3.0.3, the sun JDK 1.6.0_26.

        I get some classpath errors like

        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Compilation failure: Compilation failure:
        [ERROR] branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:[105,63] package org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos does not exist
        

        Radim, did you see anything like this?

        Show
        Robert Joseph Evans added a comment - branch-0.23 does not seem to build for me after applying the patch. I am using RHEL 5, mvn 3.0.3, the sun JDK 1.6.0_26. I get some classpath errors like [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Compilation failure: Compilation failure: [ERROR] branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:[105,63] package org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos does not exist Radim, did you see anything like this?
        Hide
        Radim Kolar added a comment -

        I am using branch-0.23 for production because it does not changes that fast as branch-2, so retesting before upgrading deployed snapshot is not needed. I have also set of patches for better resource management developed on top of branch-0.23; i didn't tried them on branch-2.

        You are right that branch-0.23 is subset of branch-2.

        Show
        Radim Kolar added a comment - I am using branch-0.23 for production because it does not changes that fast as branch-2, so retesting before upgrading deployed snapshot is not needed. I have also set of patches for better resource management developed on top of branch-0.23; i didn't tried them on branch-2. You are right that branch-0.23 is subset of branch-2.
        Hide
        Harsh J added a comment -

        The branch-0.23 isn't being maintained by the whole of the community anymore. It remains and continues to get fixes by maintainers at Yahoo! but all of the mainstream development is currently occurring on branch-2.

        Whats stopping you from using branch-2 at this point Radim? Does it not work on FreeBSD? Cause as I see it, branch-0.23 is now a subset of branch-2. Correct me if am wrong.

        Show
        Harsh J added a comment - The branch-0.23 isn't being maintained by the whole of the community anymore. It remains and continues to get fixes by maintainers at Yahoo! but all of the mainstream development is currently occurring on branch-2. Whats stopping you from using branch-2 at this point Radim? Does it not work on FreeBSD? Cause as I see it, branch-0.23 is now a subset of branch-2. Correct me if am wrong.
        Hide
        Radim Kolar added a comment -

        patch is for branch-0.23. QA bot tried to apply it to trunk.

        Show
        Radim Kolar added a comment - patch is for branch-0.23. QA bot tried to apply it to trunk.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12534004/hadoop-invalid-pom-023.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1156//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12534004/hadoop-invalid-pom-023.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1156//console This message is automatically generated.

          People

          • Assignee:
            Radim Kolar
            Reporter:
            Radim Kolar
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development