Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1309

I want to change the rumen job trace generator to use a more modular internal structure, to allow for more input log formats

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • tools/rumen
    • None
    • Reviewed

    Description

      There are two orthogonal questions to answer when processing a job tracker log: how will the logs and the xml configuration files be packaged, and in which release of hadoop map/reduce were the logs generated? The existing rumen only has a couple of answers to this question. The new engine will handle three answers to the version question: 0.18, 0.20 and current, and two answers to the packaging question: separate files with names derived from the job ID, and concatenated files with a header between sections [used for easier file interchange].

      Attachments

        1. demuxer-plus-concatenated-files--2009-12-21.patch
          210 kB
          Dick King
        2. demuxer-plus-concatenated-files--2010-01-06.patch
          211 kB
          Dick King
        3. demuxer-plus-concatenated-files--2010-01-08.patch
          209 kB
          Dick King
        4. demuxer-plus-concatenated-files--2010-01-08-b.patch
          208 kB
          Dick King
        5. demuxer-plus-concatenated-files--2010-01-08-c.patch
          206 kB
          Dick King
        6. demuxer-plus-concatenated-files--2010-01-08-d.patch
          208 kB
          Dick King
        7. demuxer-plus-concatenated-files--2010-01-11.patch
          209 kB
          Dick King
        8. mapreduce-1309--2009-01-14.patch
          209 kB
          Dick King
        9. mapreduce-1309--2009-01-14-a.patch
          208 kB
          Dick King
        10. mapreduce-1309--2010-01-20.patch
          208 kB
          Dick King
        11. mapreduce-1309--2010-02-03.patch
          224 kB
          Dick King
        12. mapreduce-1309--2010-02-04.patch
          225 kB
          Dick King
        13. mapreduce-1309--2010-02-10.patch
          226 kB
          Dick King
        14. mapreduce-1309--2010-02-12.patch
          226 kB
          Dick King
        15. mapreduce-1309--2010-02-16.patch
          227 kB
          Dick King
        16. mapreduce-1309--2010-02-16-a.patch
          227 kB
          Dick King
        17. mapreduce-1309--2010-02-17.patch
          226 kB
          Dick King
        18. rumen-yhadoop-20.patch
          571 kB
          Hong Tang
        19. mr-1309-yhadoop-20.10.patch
          570 kB
          Hong Tang

        Issue Links

          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-1479 The trace generator should test operation on the current format by building a test case that runs an ad hoc cluster

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-1459 There are javadoc warnings in Trunk

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              dking Dick King
              dking Dick King
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: