[BEAM-6697] ParquetIO Performance test is failing on (GCS filesystem)

• Type: New Feature
• Status: Resolved
• Priority: P0
• Resolution: Fixed
• Fix Version/s: 2.11.0
• Component/s: io-java-parquet, test-failures
• Assignee: Chamikara Madhusanka Jayalath
• Reporter: Lukasz Gajowy
• Created: 16/Feb/19 13:20
• Updated: 16/May/20 14:21
• Resolved: 26/Feb/19 09:38

Description

Relevant failure logs: 

Caused by: java.lang.RuntimeException: org.apache.beam.sdk.io.parquet.ParquetIO$ReadFiles$BeamParquetInputFile@2de8303e is not a Parquet file (too small length: -1)
    	at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:514)
    	at org.apache.parquet.hadoop.ParquetFileReader.<init>(ParquetFileReader.java:689)
    	at org.apache.parquet.hadoop.ParquetFileReader.open(ParquetFileReader.java:595)
    	at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:152)
    	at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
    	at org.apache.beam.sdk.io.parquet.ParquetIO$ReadFiles$ReadFn.processElement(ParquetIO.java:221)

 

Full logs can be found here: https://builds.apache.org/view/A-D/view/Beam/view/PerformanceTests/job/beam_PerformanceTests_ParquetIOIT/

 

 

 

Issue Links

• supercedes: BEAM-6736 Upgrade gcsio dependency to 1.9.15
• links to: GitHub Pull Request #7943