[PARQUET-528] Fix flush() for RecordConsumer and implementations

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0, 1.8.1
• Fix Version/s: 1.9.0, 1.8.2
• Component/s: parquet-mr
• Assignee: Liwei Lin(Inactive)
• Reporter: Liwei Lin(Inactive)
• Created: 14/Feb/16 14:02
• Updated: 23/Jun/24 03:28
• Resolved: 05/Mar/16 11:46

Description

flush() was added in RecordConsumer and MessageColumnIO to help implementing nulls caching.

However, other RecordConsumer implementations should also implements flush() properly. For instance, RecordConsumerLoggingWrapper and ValidatingRecordConsumer should call delegate.flush() in their flush() methods, otherwise data might be mistakenly truncated.

This ticket:

• makes flush() abstract in RecordConsumer
• implements flush() properly for all RecordConsumer subclasses, specifically:
  • RecordConsumerLoggingWrapper
  • ValidatingRecordConsumer
  • ConverterConsumer
  • ExpectationValidatingRecordConsumer

Issue Links

• blocks: PARQUET-401 Deprecate Log and move to SLF4J Logger
• links to: PR#325