Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-145

Limit the amount of partitions considered for GlobalBloomIndex

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: New
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Index, newbie
    • Labels:
      None

      Description

      Currently, global bloom index will check inputs against files in all partitions.. In lot of cases, the user may know a range of partitions actually impacted from updates clearly (e.g upstream system drops updates older than a year, ... ).. In such a scenario,it may make sense to support an option for Global bloom to control how many partitions you want to match against, to gain performance.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jerryzhao1423 jerry
                Reporter:
                vinoth Vinoth Chandar
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: