Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1201

Column indexes

    XMLWordPrintableJSON

Details

    Description

      Write the column indexes described in PARQUET-922.
      This is the first phase of implementing the whole feature. The implementation is done in the following steps:

      • Utility to read/write indexes in parquet-format
      • Writing indexes in the parquet file
      • Extend parquet-tools and parquet-cli to show the indexes
      • Limit index size based on parquet properties
      • Trim min/max values where possible based on parquet properties
      • Filtering based on column indexes

      The work is done on the feature branch column-indexes. This JIRA will be resolved after the branch has been merged to master.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-1365 Don't write page level statistics

          • Major - Major loss of function.
          • Resolved
          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-922 Add index pages to the format to support efficient page skipping

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-1415 Improve logic when to write column indexes

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-1435 Benchmark filtering column-indexes

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. PARQUET-1207 Write index page in parquet file

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-1414 Limit page size based on maximum row count

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-1739 Make Spark SQL support Column indexes

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link format PR

          Web Link GitHub Pull Request #527

          Activity

            People

              gszadovszky Gabor Szadovszky
              gszadovszky Gabor Szadovszky
              Votes:
              3 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: