Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-386

Printing out the statistics of metadata in parquet-tools

    XMLWordPrintableJSON

    Details

      Description

      While playing with "parquet-tools", I found that the statistics data of columns is not being printed out when the following is executed;

      $ java -jar parquet-tools-1.6.0rc3-SNAPSHOT.jar schema --detailed perf.1000.parquet

      And the output for a row group like this;

      =====================================================================================================================

      row group 1: RC:747388 TS:134218473 OFFSET:4
      --------------------------------------------------------------------------------
      cust_key: INT64 UNCOMPRESSED DO:0 FPO:4 SZ:5979444/5979444/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      name: BINARY UNCOMPRESSED DO:0 FPO:5979448 SZ:16443766/16443766/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      address: BINARY UNCOMPRESSED DO:0 FPO:22423214 SZ:21716568/21716568/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      nation_key: INT32 UNCOMPRESSED DO:0 FPO:44139782 SZ:2989697/2989697/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      phone: BINARY UNCOMPRESSED DO:0 FPO:47129479 SZ:14201364/14201364/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      acctbal: DOUBLE UNCOMPRESSED DO:0 FPO:61330843 SZ:5979444/5979444/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      mktsegment: BINARY UNCOMPRESSED DO:0 FPO:67310287 SZ:9714675/9714675/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED
      comment_col: BINARY UNCOMPRESSED DO:0 FPO:77024962 SZ:57193515/57193515/1.00 VC:747388 ENC:PLAIN,RLE,BIT_PACKED

      =====================================================================================================================

      However, it would be great to print out the data of statistics of metadata.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gszadovszky Gabor Szadovszky
                Reporter:
                onrsyr Onur Soyer
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: