Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-2833

Clean up unused archive files instead of expanding indefinitely

Details

    • 0.25

    Description

      As we know, most of storage do not support append action, so that hoodie will create a new archive file under archived dictionary when archiving.

      As time goes by, there may be thousands of archive files, which most of them is not useful anymore.

      Maybe it is meaningful to have a function to clean these unused archive files.

      Attachments

        Issue Links

          Activity

            lei w lei w added a comment - - edited

            hi ,zhangyue19921010, this feature is to merage all candidateFiles(small archive instant) to a new logFile, then delete the candidateFiles? I have a question is what time of the new logFile will be cleaned up?

            lei w lei w added a comment - - edited hi , zhangyue19921010 , this feature is to merage all candidateFiles(small archive instant) to a new logFile, then delete the candidateFiles? I have a question is what time of the new logFile will be cleaned up?

            People

              Unassigned Unassigned
              zhangyue19921010 Yue Zhang
              Y Ethan Guo
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 0.5h
                  0.5h
                  Remaining:
                  Remaining Estimate - 0.5h
                  0.5h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified