XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 3.4.0
    • None
    • SQL
    • None

    Description

      Currently, Spark DS V2 supports push down Limit operator to data source.
      But the behavior only controlled by pushDownList option.
      If the limit is very large, then Executor will pull all the result set from data source.
      So it will cause the memory issue as you know.

      Attachments

        Activity

          People

            Unassigned Unassigned
            beliefer Jiaan Geng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: