Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1170

YARN support for Drill

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.12.0
    • 1.13.0
    • None

    Description

      This is a tracking item to make Drill work with YARN.
      Below are few requirements/needs to consider.

      • Drill should run as an YARN based application, side by side with other YARN enabled applications (on same nodes or different nodes). Both memory and CPU resources of Drill should be controlled in this mechanism.
      • As an YARN enabled application, Drill resource consumption should be adaptive to the load on the cluster. For ex: When there is no load on the Drill , Drill should consume no resources on the cluster. As the load on Drill increases, resources permitting, usage should grow proportionally.
      • Low latency is a key requirement for Apache Drill along with support for multiple users (concurrency in 100s-1000s). This should be supported when run as YARN application as well.

      Attachments

        1. Drill-on-YARNDesignOverview.pdf
          91 kB
          Paul Rogers
        2. Drill-on-YARNUserGuide.pdf
          600 kB
          Paul Rogers

        Activity

          People

            Paul.Rogers Paul Rogers
            Neeraja Neeraja
            Salim Achouche Salim Achouche
            Votes:
            6 Vote for this issue
            Watchers:
            25 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: