Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-2032

Deprecate LZ4, introduce new LZ4_RAW

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: parquet-mr
    • Labels:
      None

      Description

      The currently implemented LZ4 compression is based on the hadoop codec which is now deprecated (see details at PARQUET-1996). Also, a new, properly specified LZ4 compression (LZ4_RAW) has been introduced in the format.

      The idea is to use the new LZ4_RAW compression for all cases when we currently use LZ4 and introduce a new configuration where the user would be able to switch to the deprecated behavior in case of the selected codec is LZ4.

      For LZ4_RAW we will need to add the proper libs that will provide the officially specified LZ4 raw format.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gszadovszky Gabor Szadovszky

              Dates

              • Created:
                Updated:

                Issue deployment