Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2214

Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: build
    • Labels:
      None
    • Environment:

      RHEL 6.1 & Ubuntu 11.04; Sun JDK 1.6_016 & Sun JDK 1.6.0_26; Ant 1.8.2

    • Hadoop Flags:
      Reviewed

      Description

      The generated poms inject the version of hdfs itsel, but hardcode the version of hadoop-common they depend on.
      When trying to build downstream projects for example mapreduce, then they will require hadoop-common-0.22.0-SNAPSHOT.jar.

      When trying to do an offline build this will fail to resolve as another hadoop-common has been installed in the local maven repo.
      Even during online build, it should compile against the hadoop-common that hdfs compiled against.

      When versions mismatch one cannot do a coherent build. That is particularly problematic when making simultaneous change in hadoop-common and hadoop-hdfs.

      1. HDFS-2214-follow-up.patch
        0.5 kB
        Joep Rottinghuis
      2. HDFS-2214.patch
        2 kB
        Joep Rottinghuis
      3. HDFS-2214.patch
        2 kB
        Konstantin Shvachko
      4. HDFS-2214.patch
        2 kB
        Konstantin Boudnik
      5. HDFS-2214.patch
        2 kB
        Joep Rottinghuis

        Issue Links

          Activity

          Joep Rottinghuis created issue -
          Joep Rottinghuis made changes -
          Field Original Value New Value
          Link This issue is related to HDFS-2211 [ HDFS-2211 ]
          Hide
          Joep Rottinghuis added a comment -

          Both this patch and HDFS-2211 are needed to make mapreduce compile offline against local version of hadoop-common.

          Show
          Joep Rottinghuis added a comment - Both this patch and HDFS-2211 are needed to make mapreduce compile offline against local version of hadoop-common.
          Joep Rottinghuis made changes -
          Attachment HDFS-2214.patch [ 12488155 ]
          Joep Rottinghuis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment -

          This patch doesn't apply cleanly to 0.22 anymore. Pl. fix and resubmit.

          Show
          Konstantin Boudnik added a comment - This patch doesn't apply cleanly to 0.22 anymore. Pl. fix and resubmit.
          Konstantin Shvachko made changes -
          Attachment HDFS-2214.patch [ 12489649 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12489649/HDFS-2214.patch
          against trunk revision 1155998.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1064//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12489649/HDFS-2214.patch against trunk revision 1155998. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1064//console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          Fixing some white space issues in the new patch.

          Show
          Konstantin Boudnik added a comment - Fixing some white space issues in the new patch.
          Konstantin Boudnik made changes -
          Attachment HDFS-2214.patch [ 12490583 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490583/HDFS-2214.patch
          against trunk revision 1158025.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1106//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490583/HDFS-2214.patch against trunk revision 1158025. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1106//console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          +1 patch looks good and works as expected.

          I have just committed it. Thanks Joep.

          Show
          Konstantin Boudnik added a comment - +1 patch looks good and works as expected. I have just committed it. Thanks Joep.
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.22.0 [ 12314241 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #76 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/76/)
          HDFS-2214. Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis.

          cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1158481
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #76 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/76/ ) HDFS-2214 . Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis. cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1158481 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Hide
          Joep Rottinghuis added a comment -

          Somehow in this patch it seems we rolled back HDFS-2189. It looks like we drooped the guava dependency again. I'll have to check today and verify.
          If indeed we did, then how to we rectify that situation?
          Do we want to roll back this patch and I attach the proper one that does not roll back the guava?

          Show
          Joep Rottinghuis added a comment - Somehow in this patch it seems we rolled back HDFS-2189 . It looks like we drooped the guava dependency again. I'll have to check today and verify. If indeed we did, then how to we rectify that situation? Do we want to roll back this patch and I attach the proper one that does not roll back the guava?
          Hide
          Joep Rottinghuis added a comment -

          Need to add back in the guava dependency that was inadvertently dropped. Will attach follow-up patch for this.

          Show
          Joep Rottinghuis added a comment - Need to add back in the guava dependency that was inadvertently dropped. Will attach follow-up patch for this.
          Joep Rottinghuis made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Joep Rottinghuis made changes -
          Attachment HDFS-2214-follow-up.patch [ 12490737 ]
          Hide
          Joep Rottinghuis added a comment -

          HDFS-2214.patch rolls back the HDFS-2189 fix which it should not have. HDFS-2214-follow-up.patch rectifies this.

          Show
          Joep Rottinghuis added a comment - HDFS-2214 .patch rolls back the HDFS-2189 fix which it should not have. HDFS-2214 -follow-up.patch rectifies this.
          Joep Rottinghuis made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Hide
          Konstantin Boudnik added a comment -

          Yes, we'd better rollback the initial patch and then apply a correct one (without manipulations with guava libs). Could you create new correct patch for that?

          Show
          Konstantin Boudnik added a comment - Yes, we'd better rollback the initial patch and then apply a correct one (without manipulations with guava libs). Could you create new correct patch for that?
          Hide
          Joep Rottinghuis added a comment -

          Ok, I will create a new patch this morning that will apply cleanly after rolling back the initial one.

          Show
          Joep Rottinghuis added a comment - Ok, I will create a new patch this morning that will apply cleanly after rolling back the initial one.
          Hide
          Joep Rottinghuis added a comment -

          Attached updated patch. Should be applied after rolling previous patch back.
          Alternatively, HDSF-2214-follow-up.patch could be applied as well. Both should end up with the same.

          Show
          Joep Rottinghuis added a comment - Attached updated patch. Should be applied after rolling previous patch back. Alternatively, HDSF-2214-follow-up.patch could be applied as well. Both should end up with the same.
          Joep Rottinghuis made changes -
          Attachment HDFS-2214.patch [ 12490864 ]
          Hide
          Konstantin Shvachko added a comment -

          Reverted and committed the new patch. Thank you Joep.

          Show
          Konstantin Shvachko added a comment - Reverted and committed the new patch. Thank you Joep.
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Joep Rottinghuis made changes -
          Assignee Joep Rottinghuis [ jrottinghuis ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #77 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/77/)
          HDFS-2214. Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis.
          HDFS-2214. Revert last patch.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160361
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160358
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #77 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/77/ ) HDFS-2214 . Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT. Contributed by Joep Rottinghuis. HDFS-2214 . Revert last patch. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160361 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1160358 Files : /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/branches/branch-0.22/hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/branches/branch-0.22/hdfs/build.xml
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          56m 38s 1 Joep Rottinghuis 29/Jul/11 00:57
          Resolved Resolved Reopened Reopened
          1d 2h 45m 1 Joep Rottinghuis 18/Aug/11 04:25
          Reopened Reopened Patch Available Patch Available
          4m 19s 1 Joep Rottinghuis 18/Aug/11 04:29
          Patch Available Patch Available Resolved Resolved
          23d 15h 41m 2 Konstantin Shvachko 22/Aug/11 19:29

            People

            • Assignee:
              Joep Rottinghuis
              Reporter:
              Joep Rottinghuis
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development