Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6814

Query performance on S3 files

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 1.14.0
    • Fix Version/s: None
    • Component/s: Storage - Other
    • Labels:
      None
    • Environment:

      Amazon EC2 instances-
      4 Linux Redhat machines -version 7.5
      RAM- 32GB

      Description

      I have installed 4 Node drill cluster on Amazon EC2 and  trying to execute a simple count on one Amazon S3 file. File type is CSV and size is approx- 14GB.
      The query returns expected count after the execution of approx 30 minutes.
      If we keep the same file in hdfs or create a table in postgres, execution time is relatively very less (approx 2-3 minutes).
      Is it normal behavior or something can be done for S3 files to make execution time comparable ?

        Attachments

        1. S3investigate.txt
          11 kB
          Denys Ordynskiy

          Activity

            People

            • Assignee:
              denysord88 Denys Ordynskiy
              Reporter:
              ashishkshukladb Ashish Shukla
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: