Pig
  1. Pig
  2. PIG-2693

LoadFunc.setLocation should be called before LoadMetadata.getStatistics

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None

      Description

      We ran into a bug with Pig/HCatalog integration on the trunk. The issue is that JobControlCompiler calls the adjustNumReducers method just before it calls setLocation on all of the LoadFuncs. This causes problems, since some loaders (i.e., HCatLoader) need setLocation to be called before it can respond to getStatistics with it's data size.

      1. PIG-2693.1.patch
        1 kB
        Bill Graham
      2. PIG-2693.2.patch
        2 kB
        Bill Graham
      3. PIG-2693.3.patch
        2 kB
        Julien Le Dem

        Activity

        Bill Graham made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Julien Le Dem made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.11 [ 12318878 ]
        Resolution Fixed [ 1 ]
        Julien Le Dem made changes -
        Attachment PIG-2693.3.patch [ 12528160 ]
        Bill Graham made changes -
        Attachment PIG-2693.2.patch [ 12527531 ]
        Bill Graham made changes -
        Field Original Value New Value
        Attachment PIG-2693.1.patch [ 12526443 ]
        Bill Graham created issue -

          People

          • Assignee:
            Bill Graham
            Reporter:
            Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development