Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1623

Solid completion time on timeline

    XMLWordPrintableJSON

Details

    Description

      We suggest a new file naming for the completed metadata file:

      ${start_time}.${action}.${completion_time}

       

      We also need a global Time Generator that can ensure the monotonical increasing generation of the timestamp, for example, maybe hold a mutex lock with the last generated timestamp backing up there. Say it may holds a lock L1. For each instant time generation, it needs guard from the lock.

       

      Before creating the completed file, we also need a lock guard from L1.

       

      Things need to note:
      1. we only add completion timestamp to the completed metadata file;
      2. we only add lock guard to the completed metadata file creation, not the whole commiting procedure;
      3. for regular instant time generation, we also need a lock (that we should ship out by default)

      Attachments

        Issue Links

          Activity

            People

              danny0405 Danny Chen
              nishith29 Nishith Agarwal
              Vinoth Chandar
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: