Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-196

parquet-tools command to get rowcount & size

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.6.0
    • 1.10.0
    • parquet-mr

    Description

      Parquet files contain metadata about rowcount & file size. We should have new commands to get rows count & size.
      These command can be added in parquet-tools:
      1. rowcount : This should add number of rows in all footers to give total rows in data.
      2. size : This should give compresses size in bytes and human readable format too.
      These command helps us to avoid parsing job logs or loading data once again to find number of rows in data. This comes very handy in complex processes, stats generation, QA etc..

      Attachments

        Activity

          People

            Unassigned Unassigned
            swapnilushinde Swapnil
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 10m
                10m
                Remaining:
                Remaining Estimate - 10m
                10m
                Logged:
                Time Spent - Not Specified
                Not Specified