Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-796

Delta Encoding is not used when dictionary enabled

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.9.0
    • None
    • parquet-mr
    • None

    Description

      Current code doesn't enable using both Delta Encoding and Dictionary Encoding. If I instantiate ParquetWriter like this : 

      val writer = new ParquetWriter[Group](outFile, new GroupWriteSupport, codec, blockSize, pageSize, dictPageSize, enableDictionary = true, true, ParquetProperties.WriterVersion.PARQUET_2_0, configuration)

      Then this piece of code :
      https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/column/values/factory/DefaultValuesWriterFactory.java#L78-L86

      Causes that DictionaryValuesWriter is used instead of the inferred DeltaLongEncodingWriter.

      The original issue is here : https://github.com/apache/parquet-mr/pull/154#issuecomment-266489768

      Attachments

        Activity

          People

            Unassigned Unassigned
            l154k Jakub Liska
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: