Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-196

parquet-tools command to get rowcount & size

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.10.0
    • Component/s: parquet-mr
    • Labels:

      Description

      Parquet files contain metadata about rowcount & file size. We should have new commands to get rows count & size.
      These command can be added in parquet-tools:
      1. rowcount : This should add number of rows in all footers to give total rows in data.
      2. size : This should give compresses size in bytes and human readable format too.
      These command helps us to avoid parsing job logs or loading data once again to find number of rows in data. This comes very handy in complex processes, stats generation, QA etc..

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              swapnilushinde Swapnil
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 10m
                10m
                Remaining:
                Remaining Estimate - 10m
                10m
                Logged:
                Time Spent - Not Specified
                Not Specified