Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-599

Hive IO dependency issues with some Hadoop profiles

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Hey folks. I was rebasing GIRAPH-13 for all the new changes today and now this happens:

      [INFO] ------------------------------------------------------------------------
      [INFO] Building Apache Giraph Hive I/O 0.2-SNAPSHOT
      [INFO] ------------------------------------------------------------------------
      Downloading: http://repo1.maven.org/maven2/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.pom
      Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.pom
      Downloading: https://repository.apache.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.pom
      Downloading: https://oss.sonatype.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.pom
      Downloaded: https://oss.sonatype.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.pom (15 KB at 20.2 KB/sec)
      Downloading: http://repo1.maven.org/maven2/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.pom
      Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.pom
      Downloading: https://repository.apache.org/content/groups/public/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.pom
      Downloading: https://oss.sonatype.org/content/groups/public/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.pom
      [WARNING] The POM for com.facebook.hadoop:hadoop-core:jar:2.0.3-alpha is missing, no dependency information available
      Downloading: http://repo1.maven.org/maven2/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.jar
      Downloading: http://repo1.maven.org/maven2/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.jar
      Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.jar
      Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.jar
      Downloading: https://repository.apache.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.jar
      Downloading: https://repository.apache.org/content/groups/public/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.jar
      Downloading: https://oss.sonatype.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.jar
      Downloading: https://oss.sonatype.org/content/groups/public/com/facebook/hadoop/hadoop-core/2.0.3-alpha/hadoop-core-2.0.3-alpha.jar
      Downloaded: https://oss.sonatype.org/content/groups/public/com/facebook/giraph/hive/hive-io-experimental/0.5/hive-io-experimental-0.5.jar (201 KB at 194.6 KB/sec)
      [INFO] ------------------------------------------------------------------------
      [INFO] Reactor Summary:
      [INFO] 
      [INFO] Apache Giraph Parent .............................. SUCCESS [0.717s]
      [INFO] Apache Giraph Core ................................ SUCCESS [2:58.276s]
      [INFO] Apache Giraph Hive I/O ............................ FAILURE [6.455s]
      [INFO] Apache Giraph Examples ............................ SKIPPED
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 3:05.779s
      [INFO] Finished at: Thu Mar 28 14:40:17 PDT 2013
      [INFO] Final Memory: 48M/352M
      [INFO] ------------------------------------------------------------------------
      [ERROR] Failed to execute goal on project giraph-hive: Could not resolve dependencies for project org.apache.giraph:giraph-hive:jar:0.2-SNAPSHOT: Could not find artifact com.facebook.hadoop:hadoop-core:jar:2.0.3-alpha in central (http://repo1.maven.org/maven2) -> [Help 1]
      [ERROR] 
      [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
      [ERROR] Re-run Maven using the -X switch to enable full debug logging.
      [ERROR] 
      [ERROR] For more information about the errors and possible solutions, please read the following articles:
      [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
      [ERROR] 
      [ERROR] After correcting the problems, you can resume the build with the command
      [ERROR]   mvn <goals> -rf :giraph-hive
      

      I was building the YARN profile, which used to cause giraph-hive not to build (incompatible profiles or something) but now Maven tries to download jars that don't exist. I suspect running a build on trunk right now from -Phadoop_2.0.x or -Phadoop_trunk would have the same problem?

      Everyone is iterating on the Hive IO so fast right now I don't want to mess with it, so I'm posting it here for everyone else to deal with

        Attachments

        1. GIRAPH-599.patch
          2 kB
          Nitay Joffe

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              initialcontext Eli Reisman
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: