Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1575

Early detection by periodically checking last written commit & active markers

    XMLWordPrintableJSON

Details

    Description

      Check if there are more commits, try to do resolution based on its current markers, and abort for a currently running job to avoid using up resources and running a concurrent job if we already found a commit that happened in the meantime. This can give back so much of the cluster early and dramatically lower costs in the cloud.

      Attachments

        Activity

          People

            zhangyue19921010 Yue Zhang
            nishith29 Nishith Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: