[HUDI-8634] Support schema on read in file group reader-based compaction - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: In Progress
Priority: Blocker
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: 1.0.1
Component/s: None
Labels:
- pull-request-available

Description

Right now we disable the file group reader-based compaction for schema on read (see below code in HoodieCompactor). For example, this test fails if we enable the file group reader-based compaction for schema on read and remove the check `StringUtils.isNullOrEmpty(config.getInternalSchema())`: `Test alter column types`

boolean useFileGroupReaderBasedCompaction = !metaClient.isMetadataTable()
    && config.getBooleanOrDefault(HoodieReaderConfig.FILE_GROUP_READER_ENABLED)
    && compactionHandler.supportsFileGroupReader()
    && !hasBootstrapFile(operations)
    && StringUtils.isNullOrEmpty(config.getInternalSchema());