Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6415

Create a tool to combine aggregated logs into HAR files

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While we wait for YARN-2942 to become viable, it would still be great to improve the aggregated logs problem. We can write a tool that combines aggregated log files into a single HAR file per application, which should solve the too many files and too many blocks problems. See the design document for details.

      See YARN-2942 for more context.

        Attachments

        1. HAR-ableAggregatedLogs_v1.pdf
          111 kB
          Robert Kanter
        2. MAPREDUCE-6415_branch-2_prelim_001.patch
          25 kB
          Robert Kanter
        3. MAPREDUCE-6415_branch-2_prelim_002.patch
          32 kB
          Robert Kanter
        4. MAPREDUCE-6415_branch-2.001.patch
          48 kB
          Robert Kanter
        5. MAPREDUCE-6415_branch-2.002.patch
          50 kB
          Robert Kanter
        6. MAPREDUCE-6415_branch-2.003.patch
          50 kB
          Robert Kanter
        7. MAPREDUCE-6415_prelim_001.patch
          25 kB
          Robert Kanter
        8. MAPREDUCE-6415_prelim_002.patch
          32 kB
          Robert Kanter
        9. MAPREDUCE-6415.001.patch
          48 kB
          Robert Kanter
        10. MAPREDUCE-6415.002.patch
          50 kB
          Robert Kanter
        11. MAPREDUCE-6415.002.patch
          50 kB
          Robert Kanter
        12. MAPREDUCE-6415.003.patch
          49 kB
          Robert Kanter

          Issue Links

            Activity

              People

              • Assignee:
                rkanter Robert Kanter
                Reporter:
                rkanter Robert Kanter
              • Votes:
                1 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: