Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-475

Unable to build giraph-hcatalog jar with dependencies..

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build, examples, test
    • Labels:
      None

      Description

      hi,

      Actually we are facing some problems that others might be having too...... we are trying to build giraph jar with dependencies to use it for graph analysis in our project..

      last 2 months we have already spent in understanding giraph working and usages and we are really impressed... we have had success with working on incubator's 1.0 release (mirror download) for understanding.. so now we moved to 2.0.. well thats when all problems starts.. even though every thing compiles successfully and no problem in generating jar for giraph main project...

      The subproject, giraph-format-contrib,
      gives error for checkstyle's not sure how remove, so i used -Dcheckstyle.skip to ignore checksyle (may be i shudnt do that but its not working otherwise) and i was able to generate SNAPSHOT.jar and tests.jar but then no jar with dependencies...

      command :
      mvn -Phadoop_non_secure clean verify -Dcheckstyle.skip

      with following result :

      Results :

      Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

      13/01/05 13:42:03 INFO zk.ZooKeeperManager: run: Shutdown hook started.
      [INFO]
      [INFO] — maven-jar-plugin:2.3.2:jar (default-jar) @ giraph-formats-contrib —
      [INFO] Building jar: /home/dipu/Downloads/pradeep/giraph/trunk/giraph-formats-contrib/target/giraph-formats-contrib-0.2-SNAPSHOT.jar
      [INFO]
      [INFO] — maven-jar-plugin:2.3.2:test-jar (default) @ giraph-formats-contrib —
      [INFO] Building jar: /home/dipu/Downloads/pradeep/giraph/trunk/giraph-formats-contrib/target/giraph-formats-contrib-0.2-SNAPSHOT-tests.jar
      [INFO]
      [INFO] — maven-checkstyle-plugin:2.9:check (default) @ giraph-formats-contrib —
      [INFO]
      [INFO] >>> findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib >>>
      [WARNING] Failure to transfer asm:asm/maven-metadata.xml from file:../../local.repository/trunk was cached in the local repository, resolution will not be reattempted until the update interval of local.repository has elapsed or updates are forced. Original error: Could not transfer metadata asm:asm/maven-metadata.xml from/to local.repository (file:../../local.repository/trunk): No connector available to access repository local.repository (file:../../local.repository/trunk) of type legacy using the available factories WagonRepositoryConnectorFactory
      [WARNING] Failure to transfer asm:asm/maven-metadata.xml from file:../../local.repository/trunk was cached in the local repository, resolution will not be reattempted until the update interval of local.repository has elapsed or updates are forced. Original error: Could not transfer metadata asm:asm/maven-metadata.xml from/to local.repository (file:../../local.repository/trunk): No connector available to access repository local.repository (file:../../local.repository/trunk) of type legacy using the available factories WagonRepositoryConnectorFactory
      [INFO]
      [INFO] — findbugs-maven-plugin:2.5.1:findbugs (findbugs) @ giraph-formats-contrib —
      [INFO] Fork Value is true
      [java] The following classes needed for analysis were missing:
      [java] org.apache.hadoop.security.token.Token
      [java] org.apache.hadoop.security.Credentials
      [java] Missing classes: 1
      [INFO] Done FindBugs Analysis....
      [INFO]
      [INFO] <<< findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib <<<
      [INFO]
      [INFO] — findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib —
      [INFO] BugInstance size is 0
      [INFO] Error size is 0
      [INFO] No errors/warnings found
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD SUCCESS
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 1:27.588s
      [INFO] Finished at: Sat Jan 05 13:42:38 IST 2013
      [INFO] Final Memory: 39M/186M
      [INFO] ------------------------------------------------------------------------

      so to create jar with depedencies i added following part to pom.xml......

      <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-assembly-plugin</artifactId>
      <version>2.2</version>
      <configuration>
      <outputDirectory>target</outputDirectory>
      <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
      </descriptorRefs>
      </configuration>
      <executions>
      <execution>
      <id>make-assembly</id> <!-- this is used for inheritance merges -->
      <phase>compile</phase> <!-- append to the packaging phase. -->
      <goals>
      <goal>single</goal> <!-- goals == mojos -->
      </goals>
      </execution>
      </executions>
      </plugin>

      It creates dependencies jar but misses giraph-formats main classes...

      [INFO] Scanning for projects...
      [INFO]
      [INFO] ------------------------------------------------------------------------
      [INFO] Building Apache Giraph Formats 0.2-SNAPSHOT
      [INFO] ------------------------------------------------------------------------
      [INFO]
      [INFO] — maven-assembly-plugin:2.2:single (default-cli) @ giraph-formats-contrib —
      [WARNING] Cannot include project artifact: org.apache.giraph:giraph-formats-contrib:jar:0.2-SNAPSHOT; it doesn't have an associated file or directory.
      [INFO] META-INF/ already added, skipping
      [INFO] META-INF/MANIFEST.MF already added, skipping
      [INFO] org/ already added, skipping
      [INFO] org/apache/ already added, skipping

      after this successfull built......not sure how to resolve this(adding main classes)...

      cmd : mvn -Phadoop_non_secure assembly:single

      Have i done something wrong...

      -------------------------------------------------------------------------------------------------------
      Another problem, when i tried executing sample program with main giraph jar.. benchmark examples executes successfully, but unfortunately shortest path example shows failure, which use to work with incubator's 1.0 result

      looking at the log it looks like it is looking for worker......why it is looking for another worker when i specified worker as 1(master) as its single node setup..

      log are as follows :

      user@ubuntu:~/giraph/trunk/giraph/target$ hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsVertex -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -ip shortestPathsInputGraph -of org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op shortestPathsOutput -w 1

      13/01/08 20:26:47 INFO graph.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
      13/01/08 20:26:49 INFO mapred.JobClient: Running job: job_201301082016_0003
      13/01/08 20:26:50 INFO mapred.JobClient: map 0% reduce 0%
      13/01/08 20:27:10 INFO mapred.JobClient: map 50% reduce 0%
      13/01/08 20:30:15 INFO mapred.JobClient: map 0% reduce 0%
      13/01/08 20:30:16 INFO mapred.JobClient: map 50% reduce 0%
      13/01/08 20:40:52 INFO mapred.JobClient: map 0% reduce 0%
      13/01/08 20:40:54 INFO mapred.JobClient: Job complete: job_201301082016_0003
      13/01/08 20:40:54 INFO mapred.JobClient: Counters: 6
      13/01/08 20:40:54 INFO mapred.JobClient: Job Counters
      13/01/08 20:40:54 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=650069
      13/01/08 20:40:54 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
      13/01/08 20:40:54 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
      13/01/08 20:40:54 INFO mapred.JobClient: Launched map tasks=2
      13/01/08 20:40:54 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
      13/01/08 20:40:54 INFO mapred.JobClient: Failed map tasks=1

      log file :
      2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.5.0-17-generic
      2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=mapred
      2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/usr/lib/hadoop-0.20
      2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/var/lib/hadoop-0.20/cache/mapred/mapred/local/taskTracker/dipu/jobcache/job_201301082016_0001/attempt_201301082016_0001_m_000000_0/work
      2013-01-08 20:21:51,943 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:22181 sessionTimeout=60000 watcher=org.apache.giraph.graph.BspServiceMaster@2d1e233
      2013-01-08 20:21:51,964 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:21:51,967 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:21:52,061 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:22181, sessionid = 0x13c1aa558540000, negotiated timeout = 600000
      2013-01-08 20:21:52,064 INFO org.apache.giraph.graph.BspService: process: Asynchronous connection complete.
      2013-01-08 20:21:52,074 INFO org.apache.giraph.graph.GraphMapper: map: No need to do anything when not a worker
      2013-01-08 20:21:52,074 INFO org.apache.giraph.graph.GraphMapper: cleanup: Starting for MASTER_ZOOKEEPER_ONLY
      2013-01-08 20:21:52,291 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201301082016_0001/_masterElectionDir/ubuntu_00000000000' and my bid is '/_hadoopBsp/job_201301082016_0001/_masterElectionDir/ubuntu_00000000000'
      2013-01-08 20:21:52,386 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using execution handler with 8 threads after requestFrameDecoder.
      2013-01-08 20:21:52,427 INFO org.apache.giraph.comm.netty.NettyServer: start: Started server communication server: ubuntu/127.0.0.1:30000 with up to 16 threads on bind attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288 backlog = 1
      2013-01-08 20:21:52,434 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using execution handler with 8 threads after requestEncoder.
      2013-01-08 20:21:52,440 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: I am now the master!
      2013-01-08 20:21:52,619 INFO org.apache.giraph.graph.BspService: process: applicationAttemptChanged signaled
      2013-01-08 20:21:52,838 WARN org.apache.giraph.graph.BspService: process: Unknown and unprocessed event (path=/_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir, type=NodeChildrenChanged, state=SyncConnected)
      2013-01-08 20:22:22,897 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 569551 more msecs left before giving up.
      2013-01-08 20:22:22,897 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:22:53,156 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 539292 more msecs left before giving up.
      2013-01-08 20:22:53,156 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:23:23,263 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 509185 more msecs left before giving up.
      2013-01-08 20:23:23,263 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:23:53,322 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 479126 more msecs left before giving up.
      2013-01-08 20:23:53,322 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:24:23,665 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 448783 more msecs left before giving up.
      2013-01-08 20:24:23,665 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:24:53,764 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 418684 more msecs left before giving up.
      2013-01-08 20:24:53,764 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:25:23,864 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 388584 more msecs left before giving up.
      2013-01-08 20:25:23,864 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:25:53,980 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 358468 more msecs left before giving up.
      2013-01-08 20:25:53,980 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:26:24,085 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 328363 more msecs left before giving up.
      2013-01-08 20:26:24,085 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:26:54,236 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 298212 more msecs left before giving up.
      2013-01-08 20:26:54,236 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
      2013-01-08 20:26:57,613 FATAL org.apache.giraph.graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got IOException, exiting...
      java.lang.IllegalStateException: generateVertexInputSplits: Got IOException
      at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268)
      at org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568)
      at org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633)
      at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
      Caused by: java.io.IOException: No input paths specified in job
      at org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:190)
      at org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:247)
      at org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:318)
      at org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61)
      at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:266)
      ... 3 more
      2013-01-08 20:26:57,615 INFO org.apache.giraph.zk.ZooKeeperManager: run: Shutdown hook started.
      2013-01-08 20:26:57,618 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper process.
      2013-01-08 20:26:57,944 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540000, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:26:57,945 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed)

      oKeeper.delete(ZooKeeper.java:728)
      at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:302)
      at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
      at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
      at org.apache.hadoop.mapred.Child.main(Child.java:260)
      2013-01-08 20:27:23,970 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:23,971 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:23,972 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:25,859 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:25,859 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:25,861 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:26,989 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:26,989 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:26,992 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:28,151 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:28,152 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:28,153 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:28,254 WARN org.apache.giraph.zk.ZooKeeperExt: deleteExt: Connection loss on attempt 2, waiting 5000 msecs before retrying.
      org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_1
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
      at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:728)
      at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:302)
      at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
      at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
      at org.apache.hadoop.mapred.Child.main(Child.java:260)
      2013-01-08 20:27:29,787 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:29,787 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:29,789 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:31,194 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:31,194 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:31,196 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:32,528 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
      2013-01-08 20:27:32,529 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
      2013-01-08 20:27:32,530 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
      2013-01-08 20:27:33,255 ERROR org.apache.giraph.graph.GraphMapper: run: Worker failure failed on another RuntimeException, original expection will be rethrown
      java.lang.IllegalStateException: deleteExt: Failed to delete /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_1 after 3 tries!
      at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:333)
      at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
      at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
      at org.apache.hadoop.mapred.Child.main(Child.java:260)
      2013-01-08 20:27:33,268 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
      2013-01-08 20:27:33,281 WARN org.apache.hadoop.mapred.Child: Error running child
      java.lang.IllegalStateException: run: Caught an unrecoverable exception exists: Failed to check /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions after 3 tries!
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:768)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
      at org.apache.hadoop.mapred.Child.main(Child.java:260)
      Caused by: java.lang.IllegalStateException: exists: Failed to check /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions after 3 tries!
      at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
      at org.apache.giraph.graph.BspServiceWorker.startSuperstep(BspServiceWorker.java:654)
      at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:451)
      at org.apache.giraph.graph.GraphMapper.map(GraphMapper.java:540)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:739)
      ... 7 more
      2013-01-08 20:27:33,286 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
      -------------------------------------------------------------------------------------------------------------------------------------------

      i am currently using it on single node
      hadoop 0.20.2 (non secure)
      hive n hbase(installed and configured)

      cmd used :

      mvn -Phadoop_non_secure clean
      mvn -Phadoop_non_secure verify
      mvn -Phadoop_non_secure test
      mvn -Phadoop_non_secure install

      we really want to use giraph.... we are planning to use with hive but i am badly stuck at this stage and any help will be great...

      Thanks

        Attachments

        1. GiraphHCatInputFormat.java
          14 kB
          pradeep kumar
        2. HiveGiraphRunner.java
          16 kB
          pradeep kumar

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                pradeep_kumar pradeep kumar
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: