Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-3414

Hadoop: Optimize map-reduce job planning.

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.6
    • 1.7
    • hadoop
    • None

    Description

      Currently Hadoop module has inefficient map-reduce planning engine. In particular, it assigns tasks only to affinity nodes. It could lead to situation when very huge tasks is processed by a single cluster node, while other cluster nodes are idle.

      We should implement configurable map-reduce planner which will be able to utilize the whole cluster.

      Attachments

        Activity

          People

            vozerov Vladimir Ozerov
            vozerov Vladimir Ozerov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: