Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2642

StoreMetadata.storeSchema can't access files in the output directory (Hadoop 0.23)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0, 0.9.3, 0.11
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      With Hadoop 0.20/1.0, storeSchema will be called after the temporary task files have been promoted to the output directory.

      In 0.23, the timing of temporary task file promotion has changed by design, now when storeSchema is called the files have not been promoted to the output directory.

      We have existing applications that depend on the temporary task files being promoted when storeSchema is called. This is a regression from 20.

        Attachments

          Activity

            People

            • Assignee:
              thw Thomas Weise
              Reporter:
              thw Thomas Weise
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: