Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-204

Make MOR rollback idempotent and disable using rolling stats for small file selection

    XMLWordPrintableJSON

    Details

      Description

      1. Rollback assumes all fileIds that are affected by rollback to be always present. This breaks when rollback fails intermittently and is retried

      2. Sometime Rolling stats wrongly tracks file sizes to be 0. We had earlier discussed to completely revisit rolling stats. We will disable using rolling stats to prevent wrong small file selection

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vbalaji Balaji Varadarajan
                Reporter:
                vbalaji Balaji Varadarajan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m