Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-2754

Performance improvement for IncrementalRelation

    XMLWordPrintableJSON

Details

    Description

      When HoodieIncrSource is used to fetch the update from another Hudi table, the IncrementalRelation will be used to read the data. But it has a performance issue because the column pruning and predicate pushdown don't happen. As the result, Hudi reads too much useless data.

      By enabling the column pruning and predicate pushdown, the data to read is reduced dramatically.

      Attachments

        Issue Links

          Activity

            People

              Guan Jintao
              Guan Jintao
              Alexey Kudinkin, Ethan Guo
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: