Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.0.0
-
None
Description
For a large gzipped file, since they are not splittable, spark have to use only one partition task to read and decompress it. This could be very slow.
We should log for this case in driver side.
Attachments
Issue Links
- duplicates
-
SPARK-5685 Show warning when users open text files compressed with non-splittable algorithms like gzip
- Resolved
- links to