Uploaded image for project: 'Hama'
  1. Hama
  2. HAMA-750

Determine the path of partition files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.6.2
    • bsp core
    • None

    Description

      The parent directory of input file was used to determine the path of base directory for partition files. The problem is when input is multiple files.

        protected BSPJob partition(BSPJob job, int maxTasks) throws IOException {
          String inputPath = job.getConfiguration().get(Constants.JOB_INPUT_DIR);
          Path inputDir = new Path(inputPath);
          if (fs.isFile(inputDir)) {
            inputDir = inputDir.getParent();
          }
      
          Path partitionDir = new Path(inputDir + "/partitions");
      
          if (fs.exists(partitionDir)) {
            fs.delete(partitionDir, true);
          }
      

      Simply we can create partitions on temp directory. For example, /tmp/hama-partitions/{$JOB_NAME}/

      Attachments

        1. HAMA-750_v02.patch
          12 kB
          Edward J. Yoon
        2. HAMA-750.patch
          10 kB
          Edward J. Yoon

        Activity

          People

            udanax Edward J. Yoon
            udanax Edward J. Yoon
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: