Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-70

PARQUET #36: Pig Schema Storage to UDFContext

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.6.0
    • None
    • None

    Description

      https://github.com/apache/incubator-parquet-mr/pull/36

      The ParquetLoader was not storing the pig schema into the udfcontext for the full load case which causes a schema reload on the task side, erases the requested schema, and causes problems with column index access.

      This fix stores the pig schema to both the udfcontext (for task side init) and jobcontext (for TupleReadSupport) along with other properties that should be set in the loader context (required field list and column index access toggle).

      Attachments

        Activity

          People

            Unassigned Unassigned
            dweeks-netflix Daniel Weeks
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: