Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2134

ant binary-system is broken in mapreduce project.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.1
    • Fix Version/s: 0.21.1
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      herriot

      Description

      Build failed due to unable to copy the commons instrumented jar. I could see the following error in the log.

      binary-system:
      [copy] Copying 5 files to /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT

      BUILD FAILED
      /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar to copy.

      It's pointing to the wrong path to copy the file. Actually the correct path is,

      /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system

      1. MAPREDUCE-2134.patch
        1 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Vinay Kumar Thota created issue -
          Vinay Kumar Thota made changes -
          Field Original Value New Value
          Component/s test [ 12312904 ]
          Component/s build [ 12312909 ]
          Hide
          Konstantin Boudnik added a comment -

          I have just ran this on trunk and don't see the issue.

          Show
          Konstantin Boudnik added a comment - I have just ran this on trunk and don't see the issue.
          Hide
          Vinay Kumar Thota added a comment -

          Still I could see the issue even with very clean build.

          Show
          Vinay Kumar Thota added a comment - Still I could see the issue even with very clean build.
          Hide
          Ranjit Mathew added a comment -

          FWIW, ant binary-system fails like this for me with the latest trunk (rev. 1022856):

          -compile-fault-inject:
          [echo] Start weaving aspects in place
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/JobTracker.java:1466 [error] The method refreshServiceAcl(JobConf, MapReducePolicyProvider) is undefined for the type Server
          [iajc] this.interTrackerServer.refreshServiceAcl(conf, new MapReducePolicyProvider());
          [iajc] ^^^^^^^^^^
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/JobTracker.java:4361 [error] The method refreshServiceAcl(JobConf, MapReducePolicyProvider) is undefined for the type Server
          [iajc] this.interTrackerServer.refreshServiceAcl(conf, new MapReducePolicyProvider());
          [iajc] ^^^^^^^^^^^^
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/TaskTracker.java:661 [error] The method refreshServiceAcl(JobConf, PolicyProvider) is undefined for the type Server
          [iajc] this.taskReportServer.refreshServiceAcl(fConf, policyProvider);
          [iajc] ^^^^^^^^^^
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj:190 [warning] advice defined in org.apache.hadoop.mapred.JobTrackerAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj:172 [warning] advice defined in org.apache.hadoop.mapred.JobTrackerAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:119 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:103 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:91 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskAspect.aj:72 [warning] advice defined in org.apache.hadoop.mapred.TaskAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskAspect.aj:46 [warning] advice defined in org.apache.hadoop.mapred.TaskAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/MapReducePolicyProviderAspect.aj:42 [warning] advice defined in org.apache.hadoop.mapred.MapReducePolicyProviderAspect has not been applied [Xlint:adviceDidNotMatch]
          [iajc]
          [iajc]
          [iajc] 3 errors, 8 warnings

          BUILD FAILED
          /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:233: The following error occurred while executing this line:
          /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:214: The following error occurred while executing this line:
          /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:90: compile errors: 3

          Show
          Ranjit Mathew added a comment - FWIW, ant binary-system fails like this for me with the latest trunk (rev. 1022856): -compile-fault-inject: [echo] Start weaving aspects in place [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/JobTracker.java:1466 [error] The method refreshServiceAcl(JobConf, MapReducePolicyProvider) is undefined for the type Server [iajc] this.interTrackerServer.refreshServiceAcl(conf, new MapReducePolicyProvider()); [iajc] ^^^^^^^^^^ [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/JobTracker.java:4361 [error] The method refreshServiceAcl(JobConf, MapReducePolicyProvider) is undefined for the type Server [iajc] this.interTrackerServer.refreshServiceAcl(conf, new MapReducePolicyProvider()); [iajc] ^^^^^^^^^^^^ [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/java/org/apache/hadoop/mapred/TaskTracker.java:661 [error] The method refreshServiceAcl(JobConf, PolicyProvider) is undefined for the type Server [iajc] this.taskReportServer.refreshServiceAcl(fConf, policyProvider); [iajc] ^^^^^^^^^^ [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj:190 [warning] advice defined in org.apache.hadoop.mapred.JobTrackerAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/JobTrackerAspect.aj:172 [warning] advice defined in org.apache.hadoop.mapred.JobTrackerAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:119 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:103 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskTrackerAspect.aj:91 [warning] advice defined in org.apache.hadoop.mapred.TaskTrackerAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskAspect.aj:72 [warning] advice defined in org.apache.hadoop.mapred.TaskAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/TaskAspect.aj:46 [warning] advice defined in org.apache.hadoop.mapred.TaskAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/system/aop/org/apache/hadoop/mapred/MapReducePolicyProviderAspect.aj:42 [warning] advice defined in org.apache.hadoop.mapred.MapReducePolicyProviderAspect has not been applied [Xlint:adviceDidNotMatch] [iajc] [iajc] [iajc] 3 errors, 8 warnings BUILD FAILED /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:233: The following error occurred while executing this line: /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:214: The following error occurred while executing this line: /home/ranjit/src/Hadoop/Apache/mapreduce/src/test/aop/build/aop.xml:90: compile errors: 3
          Hide
          Konstantin Boudnik added a comment -

          Yes, I can see the issue with 'missing' jar file too. The patch addresses the problem. Thanks for catching it, Vinay. Please confirm that it works for you too.

          Show
          Konstantin Boudnik added a comment - Yes, I can see the issue with 'missing' jar file too. The patch addresses the problem. Thanks for catching it, Vinay. Please confirm that it works for you too.
          Konstantin Boudnik made changes -
          Attachment MAPREDUCE-2134.patch [ 12457723 ]
          Konstantin Boudnik made changes -
          Assignee Konstantin Boudnik [ cos ]
          Konstantin Boudnik made changes -
          Affects Version/s 0.21.1 [ 12315272 ]
          Konstantin Boudnik made changes -
          Link This issue is cloned as MAPREDUCE-2144 [ MAPREDUCE-2144 ]
          Hide
          Vinay Kumar Thota added a comment -

          Patch works for me.
          +1.

          Show
          Vinay Kumar Thota added a comment - Patch works for me. +1.
          Hide
          Konstantin Boudnik added a comment -

          Awesome, I'll commit it in the morning after usual validation.

          Show
          Konstantin Boudnik added a comment - Awesome, I'll commit it in the morning after usual validation.
          Hide
          Konstantin Boudnik added a comment -

          Ranjit, your problem seems to caused by some outdated artifacts left either in ivy cache or in your local m2 repository. I have ran trunk build on a pristine machine (where I have built MR before) and your issue doesn't seem to exist.

          If you keep seeing it - please feel free to open a new JIRA.

          Show
          Konstantin Boudnik added a comment - Ranjit, your problem seems to caused by some outdated artifacts left either in ivy cache or in your local m2 repository. I have ran trunk build on a pristine machine (where I have built MR before) and your issue doesn't seem to exist. If you keep seeing it - please feel free to open a new JIRA.
          Konstantin Boudnik made changes -
          Component/s test [ 12312904 ]
          Tags herriot
          Component/s build [ 12312909 ]
          Hide
          Ranjit Mathew added a comment -

          Yeah, so much for ivy/maven easing the build process.

          I nuked "~/.ivy2", applied your patch and re-built using "ant binary-system" and this time it works.
          Thanks a lot.

          +1 for your patch.

          Show
          Ranjit Mathew added a comment - Yeah, so much for ivy/maven easing the build process. I nuked "~/.ivy2", applied your patch and re-built using "ant binary-system" and this time it works. Thanks a lot. +1 for your patch.
          Hide
          Konstantin Boudnik added a comment -

          I have ran test-patch locally:

               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no new tests are needed for this patch.
               [exec]                         Also please list what manual steps were performed to verify this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec] 
               [exec]     +1 system tests framework.  The patch passed system tests framework compile.
          

          all is well. -1 on tests is there because this patch can't have any tests.

          Show
          Konstantin Boudnik added a comment - I have ran test-patch locally: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile. all is well. -1 on tests is there because this patch can't have any tests.
          Hide
          Konstantin Boudnik added a comment -

          I have just committed this to 0.21 and trunk.

          Show
          Konstantin Boudnik added a comment - I have just committed this to 0.21 and trunk.
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.1 [ 12315272 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          7d 16h 55m 1 Konstantin Boudnik 21/Oct/10 23:17

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Vinay Kumar Thota
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development