Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-351

Fail job early when there is no input

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now we get a cryptic error message when we know that there is no input!

      2012-09-28 13:45:20,410 INFO org.apache.giraph.graph.BspServiceMaster: generateInputSplits: Got 0 input splits for 100 workers
      2012-09-28 13:45:20,410 WARN org.apache.giraph.graph.BspServiceMaster: createInputSplits: Number of inputSplits=0 < 100=number of healthy processes, some workers will be not used

      But the job keeps on going and the workers get the exception

      java.lang.IllegalStateException: setup: loadVertices failed due to KeeperException
      at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:628)
      at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:370)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:570)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:632)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
      at org.apache.hadoop.mapred.Child.main(Child.java:171)
      Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /_hadoopBsp/job_201209271814.325_0001/_inputSplitDir
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
      at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
      at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
      at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:225)
      at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:320)
      at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:607)
      ... 5 more

        Attachments

        1. GIRAPH-351.patch
          1.0 kB
          Avery Ching
        2. GIRAPH-351.2.patch
          1 kB
          Avery Ching

          Activity

            People

            • Assignee:
              aching Avery Ching
              Reporter:
              aching Avery Ching
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: