Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-442

FileNotFoundException: <baseDirectory>/_zkServer is checked before it is created

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: zookeeper
    • Labels:
      None

      Description

      On running following command

      $ hadoop fs -rm -r -skipTrash /tmp/giraph/cc/_bsp/_defaultZkManagerDir/
      $ hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.23.1-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -Dmapreduce.jobtracker.address=jt1.myorg.org:port -Dgiraph.zkManagerDirectory='/tmp/giraph/cc/_bsp/_defaultZkManagerDir' -e 1 -s 3 -v -V 500000 -w 30
      

      The ZooKeeper Server fails with FileNotFoundException

      2012-12-05 11:04:26,932 INFO [main] org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Made the directory /tmp/giraph/cc/_bsp/_defaultZkManagerDir
      2012-12-05 11:04:26,935 INFO [main] org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Creating my filestamp /tmp/giraph/cc/_bsp/_defaultZkManagerDir/_task/machine.myorg.org 0
      2012-12-05 11:04:26,965 INFO [main] org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Got [machine30955.myorg.org, machine31086.myorg.org, machine31116.myorg.org, machine31228.myorg.org, machine31245.myorg.org, machine31354.myorg.org, machine31361.myorg.org, machine31405.myorg.org, machine31631.myorg.org, machine31753.myorg.org, machine31782.myorg.org, machine31845.myorg.org, machine32065.myorg.org, machine32067.myorg.org, machine32089.myorg.org, machine32217.myorg.org, machine32229.myorg.org, machine32347.myorg.org, machine32400.myorg.org, machine32434.myorg.org, machine32517.myorg.org, machine32663.myorg.org, machine32830.myorg.org, machine33208.myorg.org, machine33228.myorg.org, machine33482.myorg.org, machine33491.myorg.org, machine33765.myorg.org, machine.myorg.org] 29 hosts from 29 candidates when 1 required (polling period is 3000) on attempt 0
      2012-12-05 11:04:26,965 INFO [main] org.apache.giraph.zk.ZooKeeperManager: createZooKeeperServerList: Creating the final ZooKeeper file '/tmp/giraph/cc/_bsp/_defaultZkManagerDir/zkServerList_machine30955.myorg.org 13 '
      2012-12-05 11:04:26,972 INFO [main] org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: For task 0, got file 'zkServerList_machine30955.myorg.org 13 ' (polling period is 3000)
      2012-12-05 11:04:26,973 INFO [main] org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Found [machine30955.myorg.org, 13] 2 hosts in filename 'zkServerList_machine30955.myorg.org 13 '
      2012-12-05 11:04:26,977 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.IllegalStateException: run: Caught an unrecoverable exception java.io.FileNotFoundException: File /tmp/giraph/cc/_bsp/_defaultZkManagerDir/_zkServer does not exist.
          at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:768)
          at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
          at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:396)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
          at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
      Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: File /tmp/giraph/cc/_bsp/_defaultZkManagerDir/_zkServer does not exist.
          at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:796)
          at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:390)
          at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:737)
          ... 7 more
      Caused by: java.io.FileNotFoundException: File /tmp/giraph/cc/_bsp/_defaultZkManagerDir/_zkServer does not exist.
          at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:362)
          at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:755)
          ... 9 more
      

        Attachments

        1. giraph-442.patch
          2 kB
          Mitesh Singh Jat
        2. giraph-442.git.patch
          2 kB
          Mitesh Singh Jat
        3. GIRAPH-442.diff
          0.9 kB
          Claudio Martella
        4. GIRAPH-442.diff
          0.9 kB
          Claudio Martella
        5. GIRAPH-442.1.patch
          2 kB
          Gianmarco De Francisci Morales

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                miteshsjat Mitesh Singh Jat
              • Votes:
                1 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: