Description
hi,
Actually we are facing some problems that others might be having too...... we are trying to build giraph jar with dependencies to use it for graph analysis in our project..
last 2 months we have already spent in understanding giraph working and usages and we are really impressed... we have had success with working on incubator's 1.0 release (mirror download) for understanding.. so now we moved to 2.0.. well thats when all problems starts.. even though every thing compiles successfully and no problem in generating jar for giraph main project...
The subproject, giraph-format-contrib,
gives error for checkstyle's not sure how remove, so i used -Dcheckstyle.skip to ignore checksyle (may be i shudnt do that but its not working otherwise) and i was able to generate SNAPSHOT.jar and tests.jar but then no jar with dependencies...
command :
mvn -Phadoop_non_secure clean verify -Dcheckstyle.skip
with following result :
Results :
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
13/01/05 13:42:03 INFO zk.ZooKeeperManager: run: Shutdown hook started.
[INFO]
[INFO] — maven-jar-plugin:2.3.2:jar (default-jar) @ giraph-formats-contrib —
[INFO] Building jar: /home/dipu/Downloads/pradeep/giraph/trunk/giraph-formats-contrib/target/giraph-formats-contrib-0.2-SNAPSHOT.jar
[INFO]
[INFO] — maven-jar-plugin:2.3.2:test-jar (default) @ giraph-formats-contrib —
[INFO] Building jar: /home/dipu/Downloads/pradeep/giraph/trunk/giraph-formats-contrib/target/giraph-formats-contrib-0.2-SNAPSHOT-tests.jar
[INFO]
[INFO] — maven-checkstyle-plugin:2.9:check (default) @ giraph-formats-contrib —
[INFO]
[INFO] >>> findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib >>>
[WARNING] Failure to transfer asm:asm/maven-metadata.xml from file:../../local.repository/trunk was cached in the local repository, resolution will not be reattempted until the update interval of local.repository has elapsed or updates are forced. Original error: Could not transfer metadata asm:asm/maven-metadata.xml from/to local.repository (file:../../local.repository/trunk): No connector available to access repository local.repository (file:../../local.repository/trunk) of type legacy using the available factories WagonRepositoryConnectorFactory
[WARNING] Failure to transfer asm:asm/maven-metadata.xml from file:../../local.repository/trunk was cached in the local repository, resolution will not be reattempted until the update interval of local.repository has elapsed or updates are forced. Original error: Could not transfer metadata asm:asm/maven-metadata.xml from/to local.repository (file:../../local.repository/trunk): No connector available to access repository local.repository (file:../../local.repository/trunk) of type legacy using the available factories WagonRepositoryConnectorFactory
[INFO]
[INFO] — findbugs-maven-plugin:2.5.1:findbugs (findbugs) @ giraph-formats-contrib —
[INFO] Fork Value is true
[java] The following classes needed for analysis were missing:
[java] org.apache.hadoop.security.token.Token
[java] org.apache.hadoop.security.Credentials
[java] Missing classes: 1
[INFO] Done FindBugs Analysis....
[INFO]
[INFO] <<< findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib <<<
[INFO]
[INFO] — findbugs-maven-plugin:2.5.1:check (default) @ giraph-formats-contrib —
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:27.588s
[INFO] Finished at: Sat Jan 05 13:42:38 IST 2013
[INFO] Final Memory: 39M/186M
[INFO] ------------------------------------------------------------------------
so to create jar with depedencies i added following part to pom.xml......
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2</version>
<configuration>
<outputDirectory>target</outputDirectory>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>compile</phase> <!-- append to the packaging phase. -->
<goals>
<goal>single</goal> <!-- goals == mojos -->
</goals>
</execution>
</executions>
</plugin>
It creates dependencies jar but misses giraph-formats main classes...
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Formats 0.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] — maven-assembly-plugin:2.2:single (default-cli) @ giraph-formats-contrib —
[WARNING] Cannot include project artifact: org.apache.giraph:giraph-formats-contrib:jar:0.2-SNAPSHOT; it doesn't have an associated file or directory.
[INFO] META-INF/ already added, skipping
[INFO] META-INF/MANIFEST.MF already added, skipping
[INFO] org/ already added, skipping
[INFO] org/apache/ already added, skipping
after this successfull built......not sure how to resolve this(adding main classes)...
cmd : mvn -Phadoop_non_secure assembly:single
Have i done something wrong...
-------------------------------------------------------------------------------------------------------
Another problem, when i tried executing sample program with main giraph jar.. benchmark examples executes successfully, but unfortunately shortest path example shows failure, which use to work with incubator's 1.0 result
looking at the log it looks like it is looking for worker......why it is looking for another worker when i specified worker as 1(master) as its single node setup..
log are as follows :
user@ubuntu:~/giraph/trunk/giraph/target$ hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsVertex -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -ip shortestPathsInputGraph -of org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op shortestPathsOutput -w 1
13/01/08 20:26:47 INFO graph.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
13/01/08 20:26:49 INFO mapred.JobClient: Running job: job_201301082016_0003
13/01/08 20:26:50 INFO mapred.JobClient: map 0% reduce 0%
13/01/08 20:27:10 INFO mapred.JobClient: map 50% reduce 0%
13/01/08 20:30:15 INFO mapred.JobClient: map 0% reduce 0%
13/01/08 20:30:16 INFO mapred.JobClient: map 50% reduce 0%
13/01/08 20:40:52 INFO mapred.JobClient: map 0% reduce 0%
13/01/08 20:40:54 INFO mapred.JobClient: Job complete: job_201301082016_0003
13/01/08 20:40:54 INFO mapred.JobClient: Counters: 6
13/01/08 20:40:54 INFO mapred.JobClient: Job Counters
13/01/08 20:40:54 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=650069
13/01/08 20:40:54 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/01/08 20:40:54 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/01/08 20:40:54 INFO mapred.JobClient: Launched map tasks=2
13/01/08 20:40:54 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/01/08 20:40:54 INFO mapred.JobClient: Failed map tasks=1
log file :
2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.5.0-17-generic
2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=mapred
2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/usr/lib/hadoop-0.20
2013-01-08 20:21:51,941 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/var/lib/hadoop-0.20/cache/mapred/mapred/local/taskTracker/dipu/jobcache/job_201301082016_0001/attempt_201301082016_0001_m_000000_0/work
2013-01-08 20:21:51,943 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:22181 sessionTimeout=60000 watcher=org.apache.giraph.graph.BspServiceMaster@2d1e233
2013-01-08 20:21:51,964 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:21:51,967 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:21:52,061 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:22181, sessionid = 0x13c1aa558540000, negotiated timeout = 600000
2013-01-08 20:21:52,064 INFO org.apache.giraph.graph.BspService: process: Asynchronous connection complete.
2013-01-08 20:21:52,074 INFO org.apache.giraph.graph.GraphMapper: map: No need to do anything when not a worker
2013-01-08 20:21:52,074 INFO org.apache.giraph.graph.GraphMapper: cleanup: Starting for MASTER_ZOOKEEPER_ONLY
2013-01-08 20:21:52,291 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201301082016_0001/_masterElectionDir/ubuntu_00000000000' and my bid is '/_hadoopBsp/job_201301082016_0001/_masterElectionDir/ubuntu_00000000000'
2013-01-08 20:21:52,386 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using execution handler with 8 threads after requestFrameDecoder.
2013-01-08 20:21:52,427 INFO org.apache.giraph.comm.netty.NettyServer: start: Started server communication server: ubuntu/127.0.0.1:30000 with up to 16 threads on bind attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288 backlog = 1
2013-01-08 20:21:52,434 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using execution handler with 8 threads after requestEncoder.
2013-01-08 20:21:52,440 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: I am now the master!
2013-01-08 20:21:52,619 INFO org.apache.giraph.graph.BspService: process: applicationAttemptChanged signaled
2013-01-08 20:21:52,838 WARN org.apache.giraph.graph.BspService: process: Unknown and unprocessed event (path=/_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir, type=NodeChildrenChanged, state=SyncConnected)
2013-01-08 20:22:22,897 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 569551 more msecs left before giving up.
2013-01-08 20:22:22,897 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:22:53,156 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 539292 more msecs left before giving up.
2013-01-08 20:22:53,156 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:23:23,263 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 509185 more msecs left before giving up.
2013-01-08 20:23:23,263 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:23:53,322 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 479126 more msecs left before giving up.
2013-01-08 20:23:53,322 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:24:23,665 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 448783 more msecs left before giving up.
2013-01-08 20:24:23,665 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:24:53,764 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 418684 more msecs left before giving up.
2013-01-08 20:24:53,764 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:25:23,864 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 388584 more msecs left before giving up.
2013-01-08 20:25:23,864 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:25:53,980 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 358468 more msecs left before giving up.
2013-01-08 20:25:53,980 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:26:24,085 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 328363 more msecs left before giving up.
2013-01-08 20:26:24,085 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:26:54,236 INFO org.apache.giraph.graph.BspServiceMaster: checkWorkers: Only found 0 responses of 1 needed to start superstep -1. Reporting every30000 msecs, 298212 more msecs left before giving up.
2013-01-08 20:26:54,236 INFO org.apache.giraph.graph.BspServiceMaster: logMissingWorkersOnSuperstep: No response from partition 1 (could be master)
2013-01-08 20:26:57,613 FATAL org.apache.giraph.graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got IOException, exiting...
java.lang.IllegalStateException: generateVertexInputSplits: Got IOException
at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268)
at org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568)
at org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633)
at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
Caused by: java.io.IOException: No input paths specified in job
at org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:190)
at org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:247)
at org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:318)
at org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61)
at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:266)
... 3 more
2013-01-08 20:26:57,615 INFO org.apache.giraph.zk.ZooKeeperManager: run: Shutdown hook started.
2013-01-08 20:26:57,618 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper process.
2013-01-08 20:26:57,944 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540000, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:26:57,945 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed)
oKeeper.delete(ZooKeeper.java:728)
at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:302)
at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
2013-01-08 20:27:23,970 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:23,971 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:23,972 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:25,859 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:25,859 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:25,861 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:26,989 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:26,989 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:26,992 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:28,151 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:28,152 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:28,153 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:28,254 WARN org.apache.giraph.zk.ZooKeeperExt: deleteExt: Connection loss on attempt 2, waiting 5000 msecs before retrying.
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:728)
at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:302)
at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
2013-01-08 20:27:29,787 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:29,787 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:29,789 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:31,194 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:31,194 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:31,196 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:32,528 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:22181
2013-01-08 20:27:32,529 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:22181, initiating session
2013-01-08 20:27:32,530 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13c1aa558540001, likely server has closed socket, closing socket connection and attempting reconnect
2013-01-08 20:27:33,255 ERROR org.apache.giraph.graph.GraphMapper: run: Worker failure failed on another RuntimeException, original expection will be rethrown
java.lang.IllegalStateException: deleteExt: Failed to delete /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_1 after 3 tries!
at org.apache.giraph.zk.ZooKeeperExt.deleteExt(ZooKeeperExt.java:333)
at org.apache.giraph.graph.BspServiceWorker.unregisterHealth(BspServiceWorker.java:615)
at org.apache.giraph.graph.BspServiceWorker.failureCleanup(BspServiceWorker.java:629)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:758)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
2013-01-08 20:27:33,268 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-01-08 20:27:33,281 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception exists: Failed to check /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions after 3 tries!
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:768)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
Caused by: java.lang.IllegalStateException: exists: Failed to check /_hadoopBsp/job_201301082016_0001/_applicationAttemptsDir/0/_superstepDir/-1/_addressesAndPartitions after 3 tries!
at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
at org.apache.giraph.graph.BspServiceWorker.startSuperstep(BspServiceWorker.java:654)
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:451)
at org.apache.giraph.graph.GraphMapper.map(GraphMapper.java:540)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:739)
... 7 more
2013-01-08 20:27:33,286 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
-------------------------------------------------------------------------------------------------------------------------------------------
i am currently using it on single node
hadoop 0.20.2 (non secure)
hive n hbase(installed and configured)
cmd used :
mvn -Phadoop_non_secure clean
mvn -Phadoop_non_secure verify
mvn -Phadoop_non_secure test
mvn -Phadoop_non_secure install
we really want to use giraph.... we are planning to use with hive but i am badly stuck at this stage and any help will be great...
Thanks
Attachments
Attachments
Issue Links
- relates to
-
GIRAPH-478 Bring back jar-with-deps for giraph-hcatalog
-
- Resolved
-