Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1219

Extra listStatus call to the namenode in WeightedRangePartitioner

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: impl
    • Labels:
      None

      Description

      We call FileLocalizer.openDFSFile in WeightedRangePartitioner to open quantile file. openDFSFile internally will check the existence of the quantile file, which adds burden to hdfs namenode. We shall remove this extra check.

        Attachments

        1. PIG-1219-3.patch
          6 kB
          Daniel Dai
        2. PIG-1219-2.patch
          5 kB
          Daniel Dai
        3. PIG-1219-1.patch
          5 kB
          Daniel Dai

          Activity

            People

            • Assignee:
              daijy Daniel Dai
              Reporter:
              daijy Daniel Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: