Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4825

Separate MapredWork into MapWork and ReduceWork

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now all the information needed to run an MR job is captured in MapredWork. This class has aliases, tagging info, table descriptors etc.

      For Tez and MRR it will be useful to break this into map and reduce specific pieces. The separation is natural and I think has value in itself, it makes the code easier to understand. However, it will also allow us to reuse these abstractions in Tez where you'll have a graph of these instead of just 1M and 0-1R.

        Attachments

        1. HIVE-4825.1.patch
          361 kB
          Gunther Hagleitner
        2. HIVE-4825.2.code.patch
          200 kB
          Gunther Hagleitner
        3. HIVE-4825.2.testfiles.patch
          1.71 MB
          Gunther Hagleitner
        4. HIVE-4825.3.testfiles.patch
          1.71 MB
          Gunther Hagleitner
        5. HIVE-4825.4.patch
          1.90 MB
          Gunther Hagleitner
        6. HIVE-4825.5.patch
          1.90 MB
          Gunther Hagleitner
        7. HIVE-4825.6.patch
          1.91 MB
          Gunther Hagleitner

          Issue Links

            Activity

              People

              • Assignee:
                hagleitn Gunther Hagleitner
                Reporter:
                hagleitn Gunther Hagleitner
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: