Uploaded image for project: 'HCatalog'
  1. HCatalog
  2. HCATALOG-373

ProgressReporter should work with both old and new MR API

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.4.1
    • None
    • None

    Description

      org.apache.hcatalog.mapreduce.ProgressReporter currently implements org.apache.hadoop.mapred.Reporter. It should also extend org.apache.hadoop.mapreduce.StatusReporter so it works with code expecting either an old or new API reporter.

      The use case is using a wrapper so a serde works with a new-API input format.

      https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java#L163

      Attachments

        1. HCATALOG-373_progress_reporter.diff
          3 kB
          Travis Crawford
        2. HCATALOG-373_progress_reporter_4.diff
          14 kB
          Travis Crawford
        3. HCATALOG-373_progress_reporter_3.diff
          15 kB
          Travis Crawford
        4. HCATALOG-373_progress_reporter_2.diff
          3 kB
          Travis Crawford

        Activity

          People

            traviscrawford Travis Crawford
            traviscrawford Travis Crawford
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: