Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-3414

Hadoop: Optimize map-reduce job planning.

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.6
    • Fix Version/s: 1.7
    • Component/s: hadoop
    • Labels:
      None

      Description

      Currently Hadoop module has inefficient map-reduce planning engine. In particular, it assigns tasks only to affinity nodes. It could lead to situation when very huge tasks is processed by a single cluster node, while other cluster nodes are idle.

      We should implement configurable map-reduce planner which will be able to utilize the whole cluster.

        Attachments

          Activity

            People

            • Assignee:
              vozerov Vladimir Ozerov
              Reporter:
              vozerov Vladimir Ozerov
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: