Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9998

Investigate updating zstd version

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 4.0.0
    • Impala 4.0.0
    • Backend
    • ghx-label-4

    Description

      Impala currently uses zstd version 1.4.0. It looks like there are some performance improvements in more recent versions:

      v1.4.5
      perf: Improved decompression speed: x64 : +10% (clang) / +5% (gcc); ARM : from +15% to +50%, depending on SoC, by @terrelln
      perf: Automatically downsizes ZSTD_DCtx when too large for too long (#2069, by @bimbashreshta)
      perf: Improved fast compression speed on aarch64 (#2040, ~+3%, by @caoyzh)
      perf: Small level 1 compression speed gains (depending on compiler)
      
      v1.4.4
      perf: Improved decompression speed, by > 10%, by @terrelln
      perf: Better compression speed when re-using a context, by @felixhandte
      perf: Fix compression ratio when compressing large files with small dictionary, by @senhuang42
      perf: zstd reference encoder can generate RLE blocks, by @bimbashrestha
      perf: minor generic speed optimization, by @davidbolvansky
      
      v1.4.1
      perf: Improve decode speed by ~7% @mgrice (#1668)
      perf: Slightly improved compression ratio of level 3 and 4 (ZSTD_dfast) by @cyan4973 (#1681)
      perf: Slightly faster compression speed when re-using a context by @cyan4973 (#1658)
      perf: Improve compression ratio for small windowLog by @cyan4973 (#1624)
      perf: Faster compression speed in high compression mode for repetitive data by @terrelln (#1635)

      https://github.com/facebook/zstd/blob/dev/CHANGELOG

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment