Uploaded image for project: 'Hama'
  1. Hama
  2. HAMA-756

Timing issue and file merging algorithm in PartitioningRunner make job fail

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 0.6.2
    • None
    • None

    Description

      There are two major problems in bsp methor of PartitioningRunner may make the partitioning fail:
      1. The call to peer.getNumPeers() may trigger the timing issue. In the special situation when some tasks complete the bsp call but some others just enter the "for (FileStatus statu : status)" loop, these remaining task calling to peer.getNumPeers() will trigger the problem.
      2. The algorithm of merging the sequence files has the problem: e.g. when desiredNum is 8 and partitioning task number (peer.getNumPeers()) is 6, the part-7 directory can not find the handler to merging it as a file.

      Attachments

        1. HAMA-756.patch
          1 kB
          MaoYuan Xian

        Activity

          People

            kennethxian MaoYuan Xian
            kennethxian MaoYuan Xian
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: