[CRUNCH-57] Add a length function to PCollection

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.4.0
• Component/s: Core
• Assignee: Josh Wills
• Reporter: Kiyan Ahmadizadeh
• Created: 05/Sep/12 21:00
• Updated: 16/Nov/12 20:06
• Resolved: 14/Sep/12 12:46

Description

Sometimes it's useful and interesting to compute the number of elements in a PCollection.

For example, suppose there was an initial PCollection that was then filtered into another. If I'm interested in how many elements of the original PCollection matched the filter, I'll have to write extra code to compute this.

PCollections should have a length method that, when called, computes the number of elements in the PCollection and returns the result.

Issue Links

• is related to: CRUNCH-58 Implement PObject in Crunch/Scrunch