Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-869

0.21.0 - snapshot incorrect dependency published in .pom files

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The snapshot available here at -
      https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0-SNAPSHOT/ , has an incorrect dependency specified for hadoop-core ( in the .pom file ).
      The source code ( in branch-0.21 ) refers to hadoop-core-0.21 in ivy/libaries.properties but pom.xml published in the repository refers to 0.22 . Please fix the same by republishing a .pom again (that points to 0.21.0 of hadoop-core , as intended ).

      1. hdfs-869.patch
        0.5 kB
        Giridharan Kesavan

        Issue Links

          Activity

          Karthik K created issue -
          Giridharan Kesavan made changes -
          Field Original Value New Value
          Assignee Giridharan Kesavan [ gkesavan ]
          Giridharan Kesavan made changes -
          Attachment hdfs-869.patch [ 12429324 ]
          Giridharan Kesavan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429324/hdfs-869.patch
          against trunk revision 894233.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429324/hdfs-869.patch against trunk revision 894233. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/166/console This message is automatically generated.
          Hide
          Karthik K added a comment -

          Can we have the latest snapshot uploaded to apache snapshot repository as well. Thanks.

          Show
          Karthik K added a comment - Can we have the latest snapshot uploaded to apache snapshot repository as well. Thanks.
          Hide
          Konstantin Boudnik added a comment -

          Snapshots are being published as the result of the successful trunk builds

          Show
          Konstantin Boudnik added a comment - Snapshots are being published as the result of the successful trunk builds
          Hide
          Karthik K added a comment -

          Thanks - Konstantin. The last build on hdfs-0.21 in hudson seems to have failed due to some environmental issues. Can we help get a latest build kicked off to see if that succeeds.

          Show
          Karthik K added a comment - Thanks - Konstantin. The last build on hdfs-0.21 in hudson seems to have failed due to some environmental issues. Can we help get a latest build kicked off to see if that succeeds.
          Hide
          stack added a comment -

          Canceling so can resubmit to see if failure on hudson was because of transient issues or indeed there is a dependency in 0.21 hadoop on 0.22.

          Show
          stack added a comment - Canceling so can resubmit to see if failure on hudson was because of transient issues or indeed there is a dependency in 0.21 hadoop on 0.22.
          stack made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          stack added a comment -

          Resubmit of patch.

          Show
          stack added a comment - Resubmit of patch.
          stack made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Karthik K made changes -
          Link This issue blocks MAPREDUCE-1352 [ MAPREDUCE-1352 ]
          Hide
          Karthik K added a comment -

          Build 168 running against hadoop trunk.

          What we may want is to run against - http://svn.apache.org/repos/asf/hadoop/hdfs/branches/branch-0.21/ , and get a green build off the same.

          Show
          Karthik K added a comment - Build 168 running against hadoop trunk. What we may want is to run against - http://svn.apache.org/repos/asf/hadoop/hdfs/branches/branch-0.21/ , and get a green build off the same.
          Hide
          stack added a comment -

          @Konstantin Is there a way to schedule a patch against Hadoop-Hdfs-21-Build? According to hudson it runs twice a day. Does that mean I need to commit this patch for it to make it into that build? Thanks boss.

          Show
          stack added a comment - @Konstantin Is there a way to schedule a patch against Hadoop-Hdfs-21-Build? According to hudson it runs twice a day. Does that mean I need to commit this patch for it to make it into that build? Thanks boss.
          Hide
          Konstantin Boudnik added a comment -

          I think there's no way you can schedule a patch to be verified for 0.21 (it isn't configured this way). What you can do is to run test-patch locally against 0.21 workspace and paste the results here.

          Show
          Konstantin Boudnik added a comment - I think there's no way you can schedule a patch to be verified for 0.21 (it isn't configured this way). What you can do is to run test-patch locally against 0.21 workspace and paste the results here.
          Hide
          Karthik K added a comment -

          The patch is to do with invalid configuration of publishing the pom file.

          The property in libraries.properties already points to 0.21-snapshot and this patch corrects that discrepancy. I am not sure if we are going to gain anything by running compile / test on this.

          @Konstantin: Can you review the patch and help commit this for dependent projects to validate. Thanks.

          Show
          Karthik K added a comment - The patch is to do with invalid configuration of publishing the pom file. The property in libraries.properties already points to 0.21-snapshot and this patch corrects that discrepancy. I am not sure if we are going to gain anything by running compile / test on this. @Konstantin: Can you review the patch and help commit this for dependent projects to validate. Thanks.
          Hide
          Karthik K added a comment -
          Snapshots are being published as the result of the successful trunk builds

          Just noticed that you are talking about trunk builds here. What we need is to get a republish of branch-0.21 build in hudson (and push the artifacts to 0.21.0 snapshot) to fix a crazy dynamic link issue in downstream systems ( hbase / mapreduce ) between different hadoop-core versions. Can you help take care of this.

          Show
          Karthik K added a comment - Snapshots are being published as the result of the successful trunk builds Just noticed that you are talking about trunk builds here. What we need is to get a republish of branch-0.21 build in hudson (and push the artifacts to 0.21.0 snapshot) to fix a crazy dynamic link issue in downstream systems ( hbase / mapreduce ) between different hadoop-core versions. Can you help take care of this.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429324/hdfs-869.patch
          against trunk revision 894233.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429324/hdfs-869.patch against trunk revision 894233. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/168/console This message is automatically generated.
          Hide
          stack added a comment -

          Looks like the resubmit passed on TRUNK so patch is innocuous in this context at least.

          @Konstantin I'll run test-patch locally and if all is well, will commit.

          @Kay Kay After commit, the 0.21 branch build should publish to the repo – I see mvn-deploy in the build.xml so my guess is that hudson will call it on success. Lets see.

          Show
          stack added a comment - Looks like the resubmit passed on TRUNK so patch is innocuous in this context at least. @Konstantin I'll run test-patch locally and if all is well, will commit. @Kay Kay After commit, the 0.21 branch build should publish to the repo – I see mvn-deploy in the build.xml so my guess is that hudson will call it on success. Lets see.
          Hide
          Karthik K added a comment -

          Thanks Konstantin / stack .

          @stack - while committing - please make sure to commit onto branch-0.21 . Looking forward to seeing a revised artifact published to hdfs -0.21.0 repository .

          Show
          Karthik K added a comment - Thanks Konstantin / stack . @stack - while committing - please make sure to commit onto branch-0.21 . Looking forward to seeing a revised artifact published to hdfs -0.21.0 repository .
          Hide
          stack added a comment -

          I ran test-patch on 0.21 branch using following command:

          ... and after adding forrest.properties to user.dir that disabled schema validation:

          ...
          ##############
          # validation properties
          
          # This set of properties determine if validation is performed
          # Values are inherited unless overridden.
          # e.g. if forrest.validate=false then all others are false unless set to true.
          forrest.validate=false
          ...
          
          ..without above forrest was off to w3c looking for schemas to validate against and failing.
          
          Here is output:
          
          

          Here is what I got...

          ...
               [exec] findbugs:
               [exec]     [mkdir] Created dir: /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs
               [exec]  [findbugs] Executing findbugs from ant task
               [exec]  [findbugs] Running FindBugs...
               [exec]  [findbugs] Calculating exit code...
               [exec]  [findbugs] Exit code set to: 0
               [exec]  [findbugs] Output saved to /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.xml
               [exec]      [xslt] Processing /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.xml to /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.html
               [exec]      [xslt] Loading stylesheet /Users/stack/bin/findbugs-1.3.9/src/xsl/default.xsl
               [exec] 
               [exec] BUILD SUCCESSFUL
               [exec] Total time: 1 minute 10 seconds
               [exec] Starting with /Users/stack/tmp/trunkFindbugsWarnings.xml
               [exec] Merging /Users/stack/tmp/patchFindbugsWarnings.xml
               [exec] 
               [exec] 
               [exec] ======================================================================
               [exec] ======================================================================
               [exec]     Determining number of patched release audit warnings.
               [exec] ======================================================================
               [exec] ======================================================================
               [exec] 
               [exec] 
               [exec] /usr/share/ant/bin/ant -Dversion=PATCH-hdfs-869.patch -Djava5.home=/System/Library/Frameworks/JavaVM.framework/Versions/1.5/Home/ -Dforrest.home=/Users/stack/bin/apache-forrest-0.8 -DHadoopPatchProcess= releaseaudit > /Users/stack/tmp/patchReleaseAuditWarnings.txt 2>&1
               [exec] 
               [exec] 
               [exec] There appear to be 102 release audit warnings before the patch and 102 release audit warnings after applying the patch.
               [exec] 
               [exec] 
               [exec] 
               [exec] 
               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no new tests are needed for this patch.
               [exec]                         Also please list what manual steps were performed to verify this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec] 
               [exec] 
               [exec] 
               [exec] 
               [exec] ======================================================================
               [exec] ======================================================================
               [exec]     Finished build.
               [exec] ======================================================================
               [exec] ======================================================================
               [exec] 
               [exec] 
          
          BUILD FAILED
          /Users/stack/checkouts/hdfs/branches/branch-0.21/build.xml:1155: exec returned: 1
          
          Show
          stack added a comment - I ran test-patch on 0.21 branch using following command: ... and after adding forrest.properties to user.dir that disabled schema validation: ... ############## # validation properties # This set of properties determine if validation is performed # Values are inherited unless overridden. # e.g. if forrest.validate= false then all others are false unless set to true . forrest.validate= false ... ..without above forrest was off to w3c looking for schemas to validate against and failing. Here is output: Here is what I got... ... [exec] findbugs: [exec] [mkdir] Created dir: /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs [exec] [findbugs] Executing findbugs from ant task [exec] [findbugs] Running FindBugs... [exec] [findbugs] Calculating exit code... [exec] [findbugs] Exit code set to: 0 [exec] [findbugs] Output saved to /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.xml [exec] [xslt] Processing /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.xml to /Users/stack/checkouts/hdfs/branches/branch-0.21/build/test/findbugs/hadoop-findbugs-report.html [exec] [xslt] Loading stylesheet /Users/stack/bin/findbugs-1.3.9/src/xsl/ default .xsl [exec] [exec] BUILD SUCCESSFUL [exec] Total time: 1 minute 10 seconds [exec] Starting with /Users/stack/tmp/trunkFindbugsWarnings.xml [exec] Merging /Users/stack/tmp/patchFindbugsWarnings.xml [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Determining number of patched release audit warnings. [exec] ====================================================================== [exec] ====================================================================== [exec] [exec] [exec] /usr/share/ant/bin/ant -Dversion=PATCH-hdfs-869.patch -Djava5.home=/ System /Library/Frameworks/JavaVM.framework/Versions/1.5/Home/ -Dforrest.home=/Users/stack/bin/apache-forrest-0.8 -DHadoopPatchProcess= releaseaudit > /Users/stack/tmp/patchReleaseAuditWarnings.txt 2>&1 [exec] [exec] [exec] There appear to be 102 release audit warnings before the patch and 102 release audit warnings after applying the patch. [exec] [exec] [exec] [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ====================================================================== [exec] [exec] BUILD FAILED /Users/stack/checkouts/hdfs/branches/branch-0.21/build.xml:1155: exec returned: 1
          Hide
          stack added a comment -

          Applied to 0.21 branch. Thanks for the patch Kay Kay.

          Show
          stack added a comment - Applied to 0.21 branch. Thanks for the patch Kay Kay.
          stack made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          stack added a comment -

          Pardon me, looks like patch was by Giridharan Kesavan. Thanks Gridharan.

          Show
          stack added a comment - Pardon me, looks like patch was by Giridharan Kesavan. Thanks Gridharan.
          Hide
          Karthik K added a comment -

          Thanks Giri / stack / Konstantin for helping with this.

          Show
          Karthik K added a comment - Thanks Giri / stack / Konstantin for helping with this.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-1352 [ MAPREDUCE-1352 ]
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-1352 [ MAPREDUCE-1352 ]

            People

            • Assignee:
              Giridharan Kesavan
              Reporter:
              Karthik K
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development