Bigtop
  1. Bigtop
  2. BIGTOP-564

Build failure for big-top trunk for pig, oozie, and whirr

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Ubuntu

      Description

      Pig:
      =====================
      [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
      [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
      [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
      [ivy:resolve] :: org.apache.hadoop#hadoop-annotations;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-auth;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-common;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-hdfs;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-core;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server-tests;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-app;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-shuffle;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-common;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-api;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-common;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server-web-proxy;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server-common;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server-nodemanager;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-yarn-server-resourcemanager;0.23.1-SNAPSHOT: not found
      [ivy:resolve] :: org.apache.hadoop#hadoop-mapreduce-client-hs;0.23.1-SNAPSHOT: not found
      [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
      [ivy:resolve]
      [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

      BUILD FAILED
      /var/lib/jenkins/jobs/bigtop-pig/workspace/output/pig/pig-0.9.2/build.xml:1476: impossible to resolve dependencies:
      resolve failed - see output for details

      Whirr:
      ======================
      [ERROR] Failed to execute goal on project whirr-cdh: Could not resolve dependencies for project org.apache.whirr:whirr-cdh:jar:0.7.1: Failure to find org.apache.thrift:thrift:jar:0.2.0 in https://repository.cloudera.com/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of cloudera has elapsed or updates are forced -> [Help 1]

      Oozie:
      ======================
      [ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:2.0-beta-6:site (default) on project oozie-docs: Execution default of goal org.apache.maven.plugins:maven-site-plugin:2.0-beta-6:site failed: Plugin org.apache.maven.plugins:maven-site-plugin:2.0-beta-6 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven.doxia:doxia-module-twiki:jar:1.0-alpha-9.2y, org.apache.maven.doxia:doxia-core:jar:1.0-alpha-9.2y: Could not find artifact org.apache.maven.doxia:doxia-module-twiki:jar:1.0-alpha-9.2y in internal.maven.repo (file:///tmp/internalmavenrepo) -> [Help 1]

      1. BIGTOP-564.patch
        3 kB
        David Liu
      2. BIGTOP-564-updated.patch
        4 kB
        David Liu
      3. BIGTOP-564-rpm.patch
        1 kB
        Adam Fokken
      4. BIGTOP-564.patch.txt
        1 kB
        Roman Shaposhnik

        Issue Links

          Activity

          David Liu created issue -
          David Liu made changes -
          Field Original Value New Value
          Link This issue is blocked by OOZIE-810 [ OOZIE-810 ]
          Hide
          David Liu added a comment -

          The oozie build issue is related to OOZIE-810.

          Show
          David Liu added a comment - The oozie build issue is related to OOZIE-810 .
          Hide
          David Liu added a comment -

          The pig and whirr build issue is because pig has a dependency on hadoop-0.23.1-SNAPSHOT, while bigtop trunk is building hadoop-2.0.0-SNAPSHOT.

          Show
          David Liu added a comment - The pig and whirr build issue is because pig has a dependency on hadoop-0.23.1-SNAPSHOT, while bigtop trunk is building hadoop-2.0.0-SNAPSHOT.
          Hide
          David Liu added a comment -

          Incorporate OOZIE-810's patch and a patch for pig's ivy/library.properties.

          Show
          David Liu added a comment - Incorporate OOZIE-810 's patch and a patch for pig's ivy/library.properties.
          David Liu made changes -
          Attachment BIGTOP-564.patch [ 12524814 ]
          Hide
          David Liu added a comment -

          Orignal patch missed a line to update the rules file. Now added. Use this instead.

          Show
          David Liu added a comment - Orignal patch missed a line to update the rules file. Now added. Use this instead.
          David Liu made changes -
          Attachment BIGTOP-564-updated.patch [ 12524816 ]
          Hide
          Peter Linnell added a comment -

          +1 LGTM Thanks for the patch!

          Show
          Peter Linnell added a comment - +1 LGTM Thanks for the patch!
          Hide
          Adam Fokken added a comment -

          Not sure which route to take as PIG-2564 addresses the pig hadoop build changes? Attached patch adds rpm resolution for pig.

          Show
          Adam Fokken added a comment - Not sure which route to take as PIG-2564 addresses the pig hadoop build changes? Attached patch adds rpm resolution for pig.
          Adam Fokken made changes -
          Attachment BIGTOP-564-rpm.patch [ 12524904 ]
          Hide
          Roman Shaposhnik added a comment -

          Wait, wait, wait, before we start patching things up – why can't we simply -D the properties?

          Show
          Roman Shaposhnik added a comment - Wait, wait, wait, before we start patching things up – why can't we simply -D the properties?
          Hide
          Roman Shaposhnik added a comment -

          Looking at it some more I think what we really need to do is to migrate to just released Pig 0.10.0. It has a much better integration with Hadoop .23 code line and brings quite a bit of goodies along.

          Now, this remained me that we haven't really discussed BOM for Bigtop 0.4.0. I'll follow up on the ML.

          Show
          Roman Shaposhnik added a comment - Looking at it some more I think what we really need to do is to migrate to just released Pig 0.10.0. It has a much better integration with Hadoop .23 code line and brings quite a bit of goodies along. Now, this remained me that we haven't really discussed BOM for Bigtop 0.4.0. I'll follow up on the ML.
          Roman Shaposhnik made changes -
          Attachment BIGTOP-564.patch.txt [ 12524939 ]
          Gavin made changes -
          Workflow no-reopen-closed, patch-avail [ 12664399 ] patch-available, re-open possible [ 12666130 ]
          Roman Shaposhnik made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee David Liu [ ap4dl ]
          Fix Version/s 0.4.0 [ 12318889 ]
          Resolution Fixed [ 1 ]
          Roman Shaposhnik made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              David Liu
              Reporter:
              David Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development