Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-535

Compatibility Issue : Hudi Readers using newer layout version can see intermittent errors due to race conditions with concurrent compactions in MOR tables

    XMLWordPrintableJSON

    Details

      Description

      As part of recent changes in 0.5.1 to avoid renames in metadata, the compaction plans are no longer getting written to .aux folder. Instead the upgraded reader reads from timeline (.hoodie). This would work as long as writer is also upgraded but if reader alone is upgraded, there would be conditions when hudi reader tried to read from compaction requested file which was renamed by concurrent compactions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vbalaji Balaji Varadarajan
                Reporter:
                vbalaji Balaji Varadarajan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m