Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-803

Provide a command line option to clean up jobtracker system directory

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: jobtracker
    • Labels:
      None

      Description

      When the JT is restarted, the mapreduce system directory's contents are used for job recovery. For sites that use this feature, there might be instances when we don't want to restart to read the mapred system directory. A sample use case is if there is a full cluster restart with a (typically minor) version upgrade of the Map/Reduce code base. To easily support such cases, it would be nice to provide a way for clean up the jobtracker system directory so that no files will be available for cleanup.

        Issue Links

          Activity

          Hide
          Hemanth Yamijala added a comment -

          The idea is loosely based on the namenode -format command. Does this make sense ?

          Show
          Hemanth Yamijala added a comment - The idea is loosely based on the namenode -format command. Does this make sense ?
          Hide
          Amar Kamat added a comment -

          Hemanth, we already have something like this today. When the jobtracker is restarted with mapred.jobtracker.restart.recover=false, the jobtracker will format the mapred.system.dir.

          Show
          Amar Kamat added a comment - Hemanth, we already have something like this today. When the jobtracker is restarted with mapred.jobtracker.restart.recover=false, the jobtracker will format the mapred.system.dir.
          Hide
          Allen Wittenauer added a comment -

          This is pretty much YARN-2131 in modern terms.

          Show
          Allen Wittenauer added a comment - This is pretty much YARN-2131 in modern terms.

            People

            • Assignee:
              Unassigned
              Reporter:
              Hemanth Yamijala
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development