Hive
  1. Hive
  2. HIVE-4825

Separate MapredWork into MapWork and ReduceWork

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now all the information needed to run an MR job is captured in MapredWork. This class has aliases, tagging info, table descriptors etc.

      For Tez and MRR it will be useful to break this into map and reduce specific pieces. The separation is natural and I think has value in itself, it makes the code easier to understand. However, it will also allow us to reuse these abstractions in Tez where you'll have a graph of these instead of just 1M and 0-1R.

      1. HIVE-4825.1.patch
        361 kB
        Gunther Hagleitner
      2. HIVE-4825.2.code.patch
        200 kB
        Gunther Hagleitner
      3. HIVE-4825.2.testfiles.patch
        1.71 MB
        Gunther Hagleitner
      4. HIVE-4825.3.testfiles.patch
        1.71 MB
        Gunther Hagleitner
      5. HIVE-4825.4.patch
        1.90 MB
        Gunther Hagleitner
      6. HIVE-4825.5.patch
        1.90 MB
        Gunther Hagleitner
      7. HIVE-4825.6.patch
        1.91 MB
        Gunther Hagleitner

        Issue Links

          Activity

          Gunther Hagleitner created issue -
          Gunther Hagleitner made changes -
          Field Original Value New Value
          Link This issue blocks HIVE-4660 [ HIVE-4660 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.1.patch [ 12591326 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.2.code.patch [ 12592282 ]
          Attachment HIVE-4825.2.testfiles.patch [ 12592283 ]
          Gunther Hagleitner made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Edward Capriolo made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Priority Major [ 3 ] Minor [ 4 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.3.testfiles.patch [ 12592505 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.4.patch [ 12593669 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.5.patch [ 12594510 ]
          Gunther Hagleitner made changes -
          Attachment HIVE-4825.6.patch [ 12594527 ]
          Ashutosh Chauhan made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Thejas M Nair made changes -
          Link This issue relates to HIVE-4955 [ HIVE-4955 ]
          Ashutosh Chauhan made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Gunther Hagleitner
              Reporter:
              Gunther Hagleitner
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development