Pig
  1. Pig
  2. PIG-2642

StoreMetadata.storeSchema can't access files in the output directory (Hadoop 0.23)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0, 0.9.3, 0.11
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      With Hadoop 0.20/1.0, storeSchema will be called after the temporary task files have been promoted to the output directory.

      In 0.23, the timing of temporary task file promotion has changed by design, now when storeSchema is called the files have not been promoted to the output directory.

      We have existing applications that depend on the temporary task files being promoted when storeSchema is called. This is a regression from 20.

      1. PIG-2642.patch
        5 kB
        Thomas Weise

        Activity

        Thomas Weise created issue -
        Thomas Weise made changes -
        Field Original Value New Value
        Attachment PIG-2642.patch [ 12522069 ]
        Thomas Weise made changes -
        Attachment PIG-2642.patch [ 12522309 ]
        Thomas Weise made changes -
        Attachment PIG-2642.patch [ 12522069 ]
        Thomas Weise made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Daniel Dai made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Thomas Weise
            Reporter:
            Thomas Weise
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development