Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2214

Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: build
    • Labels:
      None
    • Environment:

      RHEL 6.1 & Ubuntu 11.04; Sun JDK 1.6_016 & Sun JDK 1.6.0_26; Ant 1.8.2

    • Hadoop Flags:
      Reviewed

      Description

      The generated poms inject the version of hdfs itsel, but hardcode the version of hadoop-common they depend on.
      When trying to build downstream projects for example mapreduce, then they will require hadoop-common-0.22.0-SNAPSHOT.jar.

      When trying to do an offline build this will fail to resolve as another hadoop-common has been installed in the local maven repo.
      Even during online build, it should compile against the hadoop-common that hdfs compiled against.

      When versions mismatch one cannot do a coherent build. That is particularly problematic when making simultaneous change in hadoop-common and hadoop-hdfs.

      1. HDFS-2214.patch
        2 kB
        Joep Rottinghuis
      2. HDFS-2214-follow-up.patch
        0.5 kB
        Joep Rottinghuis
      3. HDFS-2214.patch
        2 kB
        Konstantin Boudnik
      4. HDFS-2214.patch
        2 kB
        Konstantin Shvachko
      5. HDFS-2214.patch
        2 kB
        Joep Rottinghuis

        Issue Links

          Activity

          Hide
          Joep Rottinghuis added a comment -

          Both this patch and HDFS-2211 are needed to make mapreduce compile offline against local version of hadoop-common.

          Show
          Joep Rottinghuis added a comment - Both this patch and HDFS-2211 are needed to make mapreduce compile offline against local version of hadoop-common.
          Hide
          Konstantin Boudnik added a comment -

          This patch doesn't apply cleanly to 0.22 anymore. Pl. fix and resubmit.

          Show
          Konstantin Boudnik added a comment - This patch doesn't apply cleanly to 0.22 anymore. Pl. fix and resubmit.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12489649/HDFS-2214.patch
          against trunk revision 1155998.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1064//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12489649/HDFS-2214.patch against trunk revision 1155998. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1064//console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          Fixing some white space issues in the new patch.

          Show
          Konstantin Boudnik added a comment - Fixing some white space issues in the new patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490583/HDFS-2214.patch
          against trunk revision 1158025.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1106//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490583/HDFS-2214.patch against trunk revision 1158025. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1106//console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          +1 patch looks good and works as expected.

          I have just committed it. Thanks Joep.

          Show
          Konstantin Boudnik added a comment - +1 patch looks good and works as expected. I have just committed it. Thanks Joep.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #76 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/76/)
          HDFS-2214. Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis.

          cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1158481
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #76 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/76/ ) HDFS-2214 . Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis. cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1158481 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Hide
          Joep Rottinghuis added a comment -

          Somehow in this patch it seems we rolled back HDFS-2189. It looks like we drooped the guava dependency again. I'll have to check today and verify.
          If indeed we did, then how to we rectify that situation?
          Do we want to roll back this patch and I attach the proper one that does not roll back the guava?

          Show
          Joep Rottinghuis added a comment - Somehow in this patch it seems we rolled back HDFS-2189 . It looks like we drooped the guava dependency again. I'll have to check today and verify. If indeed we did, then how to we rectify that situation? Do we want to roll back this patch and I attach the proper one that does not roll back the guava?
          Hide
          Joep Rottinghuis added a comment -

          Need to add back in the guava dependency that was inadvertently dropped. Will attach follow-up patch for this.

          Show
          Joep Rottinghuis added a comment - Need to add back in the guava dependency that was inadvertently dropped. Will attach follow-up patch for this.
          Hide
          Joep Rottinghuis added a comment -

          HDFS-2214.patch rolls back the HDFS-2189 fix which it should not have. HDFS-2214-follow-up.patch rectifies this.

          Show
          Joep Rottinghuis added a comment - HDFS-2214 .patch rolls back the HDFS-2189 fix which it should not have. HDFS-2214 -follow-up.patch rectifies this.
          Hide
          Konstantin Boudnik added a comment -

          Yes, we'd better rollback the initial patch and then apply a correct one (without manipulations with guava libs). Could you create new correct patch for that?

          Show
          Konstantin Boudnik added a comment - Yes, we'd better rollback the initial patch and then apply a correct one (without manipulations with guava libs). Could you create new correct patch for that?
          Hide
          Joep Rottinghuis added a comment -

          Ok, I will create a new patch this morning that will apply cleanly after rolling back the initial one.

          Show
          Joep Rottinghuis added a comment - Ok, I will create a new patch this morning that will apply cleanly after rolling back the initial one.
          Hide
          Joep Rottinghuis added a comment -

          Attached updated patch. Should be applied after rolling previous patch back.
          Alternatively, HDSF-2214-follow-up.patch could be applied as well. Both should end up with the same.

          Show
          Joep Rottinghuis added a comment - Attached updated patch. Should be applied after rolling previous patch back. Alternatively, HDSF-2214-follow-up.patch could be applied as well. Both should end up with the same.
          Hide
          Konstantin Shvachko added a comment -

          Reverted and committed the new patch. Thank you Joep.

          Show
          Konstantin Shvachko added a comment - Reverted and committed the new patch. Thank you Joep.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #77 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/77/)
          HDFS-2214. Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis.
          HDFS-2214. Revert last patch.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160361
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160358
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #77 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/77/ ) HDFS-2214 . Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis. HDFS-2214 . Revert last patch. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160361 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160358 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml

            People

            • Assignee:
              Joep Rottinghuis
              Reporter:
              Joep Rottinghuis
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development