[HAMA-756] Timing issue and file merging algorithm in PartitioningRunner make job fail

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.6.2
• Assignee: MaoYuan Xian
• Reporter: MaoYuan Xian
• Due: 13/May/13
• Created: 10/May/13 03:48
• Updated: 13/May/13 01:14
• Resolved: 12/May/13 04:02

Description

There are two major problems in bsp methor of PartitioningRunner may make the partitioning fail:
1. The call to peer.getNumPeers() may trigger the timing issue. In the special situation when some tasks complete the bsp call but some others just enter the "for (FileStatus statu : status)" loop, these remaining task calling to peer.getNumPeers() will trigger the problem.
2. The algorithm of merging the sequence files has the problem: e.g. when desiredNum is 8 and partitioning task number (peer.getNumPeers()) is 6, the part-7 directory can not find the handler to merging it as a file.