Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3974

Improve partition purning performance in presto carbon integration

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      a)  For 200K segments table in cloud, presto partition query was taking more than 5 hours. the reason is it was reading all segment files for partition pruning. Now it is less than a minute !

       

      b) If the query filter is less than or greater than or more than one value. Partition pruning is not working in presto (due to missing code in presto-carbon integration)

        Attachments

          Activity

            People

            • Assignee:
              Ajantha_Bhat Ajantha Bhat
              Reporter:
              Ajantha_Bhat Ajantha Bhat
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 8h 10m
                8h 10m