Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2230

hdfs it not resolving the latest common test jars published post common mavenization

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      hdfs it not pulling the right common test jar.
      hadoop-common test jar dependency in ivy.xml has to configure as type=tests and not as a separate module.

      1. HDFS-2230.patch
        6 kB
        Giridharan Kesavan

        Activity

        Hide
        Giridharan Kesavan added a comment -

        this patch uses ivy classifier to download the test jars published for hadoop-common.

        Show
        Giridharan Kesavan added a comment - this patch uses ivy classifier to download the test jars published for hadoop-common.
        Hide
        Eric Yang added a comment -

        +1 Works for me.

        Show
        Eric Yang added a comment - +1 Works for me.
        Hide
        Mahadev konar added a comment -

        +1 lgtm.

        Show
        Mahadev konar added a comment - +1 lgtm.
        Hide
        Giridharan Kesavan added a comment -

        I just committed this patch to hdfs trunk. thanks

        Show
        Giridharan Kesavan added a comment - I just committed this patch to hdfs trunk. thanks
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #712 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/712/)
        HDFS-2230. Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan)

        gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123
        Files :

        • /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml
        • /hadoop/common/trunk/hdfs/ivy/ivysettings.xml
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/ivy.xml
        • /hadoop/common/trunk/hdfs/build.xml
        • /hadoop/common/trunk/hdfs/ivy/libraries.properties
        • /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #712 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/712/ ) HDFS-2230 . Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan) gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123 Files : /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml /hadoop/common/trunk/hdfs/ivy/ivysettings.xml /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/ivy.xml /hadoop/common/trunk/hdfs/build.xml /hadoop/common/trunk/hdfs/ivy/libraries.properties /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #818 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/818/)
        HDFS-2230. Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan)

        gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123
        Files :

        • /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml
        • /hadoop/common/trunk/hdfs/ivy/ivysettings.xml
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/ivy.xml
        • /hadoop/common/trunk/hdfs/build.xml
        • /hadoop/common/trunk/hdfs/ivy/libraries.properties
        • /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #818 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/818/ ) HDFS-2230 . Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan) gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123 Files : /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml /hadoop/common/trunk/hdfs/ivy/ivysettings.xml /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/ivy.xml /hadoop/common/trunk/hdfs/build.xml /hadoop/common/trunk/hdfs/ivy/libraries.properties /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Some unit tests started failing. The tests passed if I reverted the patch here. Please take a look. See this comment for more details.

        Show
        Tsz Wo Nicholas Sze added a comment - Some unit tests started failing. The tests passed if I reverted the patch here. Please take a look. See this comment for more details.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The failed tests are
        TestHDFSServerPorts
        TestReplicationPolicy
        TestCheckpoint
        TestNNThroughputBenchmark
        TestValidateConfigurationSettings

        Show
        Tsz Wo Nicholas Sze added a comment - The failed tests are TestHDFSServerPorts TestReplicationPolicy TestCheckpoint TestNNThroughputBenchmark TestValidateConfigurationSettings
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #744 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/744/)
        HDFS-2230. Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan)

        gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123
        Files :

        • /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml
        • /hadoop/common/trunk/hdfs/ivy/ivysettings.xml
        • /hadoop/common/trunk/hdfs/CHANGES.txt
        • /hadoop/common/trunk/hdfs/ivy.xml
        • /hadoop/common/trunk/hdfs/build.xml
        • /hadoop/common/trunk/hdfs/ivy/libraries.properties
        • /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #744 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/744/ ) HDFS-2230 . Fix ivy conf to resolve/retrieve latest common-tests jar published by hadoop common maven build. (gkesavan) gkesavan : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1155123 Files : /hadoop/common/trunk/hdfs/src/contrib/build-contrib.xml /hadoop/common/trunk/hdfs/ivy/ivysettings.xml /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/ivy.xml /hadoop/common/trunk/hdfs/build.xml /hadoop/common/trunk/hdfs/ivy/libraries.properties /hadoop/common/trunk/hdfs/src/contrib/fuse-dfs/ivy.xml
        Hide
        Todd Lipcon added a comment -

        Strange, so somehow "test.build.data" property is ending up as "/test/", which makes MiniDFSCluster.getBaseDirectory break.

        Show
        Todd Lipcon added a comment - Strange, so somehow "test.build.data" property is ending up as "/test/", which makes MiniDFSCluster.getBaseDirectory break.
        Hide
        Giridharan Kesavan added a comment -

        @Nicholas
        hdfs-2230 patch no where touches the test.build.data property. Without this patch ivy would be resolving the common-test artifact which is couple of weeks old (before common maven patch got merged). With this patch you would be getting the latest common test artifact. Thats the difference. If the latest common test jar has the right info then this should work just fine.

        Show
        Giridharan Kesavan added a comment - @Nicholas hdfs-2230 patch no where touches the test.build.data property. Without this patch ivy would be resolving the common-test artifact which is couple of weeks old (before common maven patch got merged). With this patch you would be getting the latest common test artifact. Thats the difference. If the latest common test jar has the right info then this should work just fine.

          People

          • Assignee:
            Giridharan Kesavan
            Reporter:
            Giridharan Kesavan
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development