Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6415

Create a tool to combine aggregated logs into HAR files

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.8.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      While we wait for YARN-2942 to become viable, it would still be great to improve the aggregated logs problem. We can write a tool that combines aggregated log files into a single HAR file per application, which should solve the too many files and too many blocks problems. See the design document for details.

      See YARN-2942 for more context.

      Attachments

        1. HAR-ableAggregatedLogs_v1.pdf
          111 kB
          Robert Kanter
        2. MAPREDUCE-6415_branch-2_prelim_001.patch
          25 kB
          Robert Kanter
        3. MAPREDUCE-6415_branch-2_prelim_002.patch
          32 kB
          Robert Kanter
        4. MAPREDUCE-6415_branch-2.001.patch
          48 kB
          Robert Kanter
        5. MAPREDUCE-6415_branch-2.002.patch
          50 kB
          Robert Kanter
        6. MAPREDUCE-6415_branch-2.003.patch
          50 kB
          Robert Kanter
        7. MAPREDUCE-6415_prelim_001.patch
          25 kB
          Robert Kanter
        8. MAPREDUCE-6415_prelim_002.patch
          32 kB
          Robert Kanter
        9. MAPREDUCE-6415.001.patch
          48 kB
          Robert Kanter
        10. MAPREDUCE-6415.002.patch
          50 kB
          Robert Kanter
        11. MAPREDUCE-6415.002.patch
          50 kB
          Robert Kanter
        12. MAPREDUCE-6415.003.patch
          49 kB
          Robert Kanter

        Issue Links

          Activity

            People

              rkanter Robert Kanter
              rkanter Robert Kanter
              Votes:
              1 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: