Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1461

Feature to instruct rumen-folder utility to skip jobs worth of specific duration

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: tools/rumen
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added a ''-starts-after' option to Rumen's Folder utility. The time duration specified after the '-starts-after' option is an offset with respect to the submit time of the first job in the input trace. Jobs in the input trace having a submit time (relative to the first job's submit time) lesser than the specified offset will be ignored.
      Show
      Added a ''-starts-after' option to Rumen's Folder utility. The time duration specified after the '-starts-after' option is an offset with respect to the submit time of the first job in the input trace. Jobs in the input trace having a submit time (relative to the first job's submit time) lesser than the specified offset will be ignored.
    • Tags:
      rumen, folder, skip-jobs

      Description

      JSON outputs of rumen on production logs can be huge in the order of multiple GB. Rumen's folder utility helps in getting a smaller snapshot of this JSON data.
      It would be helpful to have an option in rumen-folder, wherein user can specify a duration from which rumen-folder should start processing data.

      Related JIRA link: https://issues.apache.org/jira/browse/MAPREDUCE-1295

      1. mr-1461-trunk-with-testcases.patch
        8 kB
        Rajesh Balamohan
      2. MR-1461-trunk.patch
        6 kB
        Rajesh Balamohan
      3. mapreduce-1461--2010-03-04.patch
        2 kB
        Rajesh Balamohan
      4. mapreduce-1461--2010-02-05.patch
        2 kB
        Rajesh Balamohan

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Amar Kamat made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Release Note Added a ''-starts-after' option to Rumen's Folder utility. The time duration specified after the '-starts-after' option is an offset with respect to the submit time of the first job in the input trace. Jobs in the input trace having a submit time (relative to the first job's submit time) lesser than the specified offset will be ignored.
        Assignee Rajesh Balamohan [ rajesh.balamohan ]
        Fix Version/s 0.23.0 [ 12315570 ]
        Resolution Fixed [ 1 ]
        Tags rumen, folder, skip-jobs
        Rajesh Balamohan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.23.0 [ 12315570 ]
        Rajesh Balamohan made changes -
        Attachment mr-1461-trunk-with-testcases.patch [ 12477476 ]
        Rajesh Balamohan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Rajesh Balamohan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Rajesh Balamohan made changes -
        Attachment MR-1461-trunk.patch [ 12475669 ]
        Nigel Daley made changes -
        Fix Version/s 0.22.0 [ 12314184 ]
        Amar Kamat made changes -
        Component/s tools/rumen [ 12313617 ]
        Rajesh Balamohan made changes -
        Attachment mapreduce-1461--2010-03-04.patch [ 12437849 ]
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hong Tang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Rajesh Balamohan made changes -
        Field Original Value New Value
        Attachment mapreduce-1461--2010-02-05.patch [ 12434931 ]
        Rajesh Balamohan created issue -

          People

          • Assignee:
            Rajesh Balamohan
            Reporter:
            Rajesh Balamohan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development