[ARROW-10134] [C++][Dataset] Add ParquetFileFragment::num_row_groups property

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: C++
• Labels: dataset dataset-dask-integration pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26145
• Assignee: Ben Kietzman
• Reporter: Joris Van den Bossche
• Created: 29/Sep/20 13:49
• Updated: 11/Jan/23 08:11
• Resolved: 08/Oct/20 00:25

Description

From https://github.com/dask/dask/pull/6534#issuecomment-699512602, comment by rjzamora:

it would be great to have access the total row-group count for the fragment from a num_row_groups attribute (which pyarrow should be able to get without parsing all row-group metadata/statistics - I think?).

One question is: does this attribute correspond to the row groups in the parquet file, or the (subset of) row groups represented by the fragment?
I expect the second (so if you do SplitByRowGroup, you would get a fragment with num_row_groups==1), but this might be a potential confusing aspect of the attribute.

Issue Links

• links to: GitHub Pull Request #8317