Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-107

Add option to disable summary metadata aggregation after MR jobs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      The summary metadata flies produced at the end of MR jobs aren't required for the job to complete successfully or for reading the data, so this should be an optional step in the MR commit phase. I'd like to add a property parquet.enable.summary-metadata that turns this off if set to false. The default will be true / enabled to match the current behavior.

        Attachments

          Activity

            People

            • Assignee:
              rdblue Ryan Blue
              Reporter:
              rdblue Ryan Blue
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: