[HAMA-949] File splits based on number of input files

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.6.4
• Fix Version/s: 0.7.0
• Assignee: Edward J. Yoon
• Reporter: Edward J. Yoon
• Created: 14/Apr/15 12:50
• Updated: 23/May/15 00:05
• Resolved: 21/May/15 00:49

Description

I've create multiple input files considering max task capacity of cluster, but it wasn't able to run. Because, currently file splits are determined based on number of blocks.

I don't know why below code has been removed. What if add this again?

    // take the short circuit path if we have already partitioned
    if (numSplits == files.length) {
      for (FileStatus file : files) {
        if (file != null) {
          splits.add(new FileSplit(file.getPath(), 0, file.getLen(),
              new String[0]));
        }
      }
      return splits.toArray(new FileSplit[splits.size()]);
    }

https://www.mail-archive.com/[e-mail]/msg00319.html