Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2693

LoadFunc.setLocation should be called before LoadMetadata.getStatistics

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None

      Description

      We ran into a bug with Pig/HCatalog integration on the trunk. The issue is that JobControlCompiler calls the adjustNumReducers method just before it calls setLocation on all of the LoadFuncs. This causes problems, since some loaders (i.e., HCatLoader) need setLocation to be called before it can respond to getStatistics with it's data size.

        Attachments

        1. PIG-2693.3.patch
          2 kB
          Julien Le Dem
        2. PIG-2693.2.patch
          2 kB
          Bill Graham
        3. PIG-2693.1.patch
          1 kB
          Bill Graham

          Activity

            People

            • Assignee:
              billgraham Bill Graham
              Reporter:
              billgraham Bill Graham
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: