Details

    • Type: Improvement Improvement
    • Status: Reopened
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.22.0
    • Fix Version/s: None
    • Component/s: tasktracker
    • Labels:
      None

      Description

      The tasktrackers cleanup the userlog directory when they restart.
      This happens independent of value of mapred.userlog.retain.hours.

      The feature is to add a configurable feature to respect mapred.userlog.retain.hours across TT restarts

        Issue Links

          Activity

          Hide
          Harsh J added a comment -

          I wonder if MAPREDUCE-2415 broke this.

          It would get cleared even if a TT is sent a reinitialization signal from the JT too. We can perhaps selectively ignore "uesrlogs/" when deleting the local dirs and leave it to the userlog-manager to clean them up per the retention policy?

          Show
          Harsh J added a comment - I wonder if MAPREDUCE-2415 broke this. It would get cleared even if a TT is sent a reinitialization signal from the JT too. We can perhaps selectively ignore "uesrlogs/" when deleting the local dirs and leave it to the userlog-manager to clean them up per the retention policy?
          Hide
          Harsh J added a comment -

          No, MAPREDUCE-2415 should not have broken this:

             public void cleanupStorage() throws IOException {
          -    this.fConf.deleteLocalFiles();
          +    this.fConf.deleteLocalFiles(SUBDIR);
          +    this.fConf.deleteLocalFiles(TT_PRIVATE_DIR);
          +    this.fConf.deleteLocalFiles(TT_LOG_TMP_DIR);
             }
          

          As we can see there, it now deletes specific sub-dirs instead of all of them, when cleaning the local dirs.

          Show
          Harsh J added a comment - No, MAPREDUCE-2415 should not have broken this: public void cleanupStorage() throws IOException { - this .fConf.deleteLocalFiles(); + this .fConf.deleteLocalFiles(SUBDIR); + this .fConf.deleteLocalFiles(TT_PRIVATE_DIR); + this .fConf.deleteLocalFiles(TT_LOG_TMP_DIR); } As we can see there, it now deletes specific sub-dirs instead of all of them, when cleaning the local dirs.
          Hide
          Arun C Murthy added a comment -

          What is the use case?

          Show
          Arun C Murthy added a comment - What is the use case?
          Hide
          Harsh J added a comment -

          Arun - Isn't this a bug? Restarts of a service should not break the log retention guarantee?

          Show
          Harsh J added a comment - Arun - Isn't this a bug? Restarts of a service should not break the log retention guarantee?
          Hide
          Harsh J added a comment -

          I could not reproduce this on 1.0.3. I restarted the TT, re-initialized it (via JT restart), but the MR local directory userlogs/ still exist and continue to be symlinked inside the logs dir (and nowhere in code can I find a full clean delete of TaskLog.USERLOGS_DIR_NAME presently). Can you post some reproduce steps for this?

          Show
          Harsh J added a comment - I could not reproduce this on 1.0.3. I restarted the TT, re-initialized it (via JT restart), but the MR local directory userlogs/ still exist and continue to be symlinked inside the logs dir (and nowhere in code can I find a full clean delete of TaskLog.USERLOGS_DIR_NAME presently). Can you post some reproduce steps for this?
          Hide
          Benoy Antony added a comment -

          Sure, I will add the steps to reproduce.

          Show
          Benoy Antony added a comment - Sure, I will add the steps to reproduce.
          Hide
          Benoy Antony added a comment -

          This issue occurs only in those distributions where MAPREDUCE-2415 is applied as well as MRAsyncDiskService is used to cleanup the volumes during TT startup.
          This is not applicable to 1.0 or 1.1 since MRAsyncDiskService is not present in those.
          I don't think , it is applicable to trunk.

          Show
          Benoy Antony added a comment - This issue occurs only in those distributions where MAPREDUCE-2415 is applied as well as MRAsyncDiskService is used to cleanup the volumes during TT startup. This is not applicable to 1.0 or 1.1 since MRAsyncDiskService is not present in those. I don't think , it is applicable to trunk.
          Hide
          Harsh J added a comment -

          Thanks very much Benoy!

          Reopen reason: This should still affect 0.22 due to MAPREDUCE-1213 being in it. Lets try to fix it for 0.22.x.

          Show
          Harsh J added a comment - Thanks very much Benoy! Reopen reason: This should still affect 0.22 due to MAPREDUCE-1213 being in it. Lets try to fix it for 0.22.x.
          Hide
          Benoy Antony added a comment -

          This will not impact 0.22. MAPREDUCE -2415 was not ported to 0.22 . So userlogs directory will not be under the scratch directories.

          Show
          Benoy Antony added a comment - This will not impact 0.22. MAPREDUCE -2415 was not ported to 0.22 . So userlogs directory will not be under the scratch directories.
          Hide
          Harsh J added a comment -

          Ah yes, my bad. Would 0.22 benefit from MAPREDUCE-2415 though? If its not needed, we can probably close this out again.

          Show
          Harsh J added a comment - Ah yes, my bad. Would 0.22 benefit from MAPREDUCE-2415 though? If its not needed, we can probably close this out again.
          Hide
          Benoy Antony added a comment -

          Good point related to porting MAPREDUCE-2415 to 0.22.
          Another related question will be porting MAPREDUCE-1213 to 1.1

          Show
          Benoy Antony added a comment - Good point related to porting MAPREDUCE-2415 to 0.22. Another related question will be porting MAPREDUCE-1213 to 1.1

            People

            • Assignee:
              Unassigned
              Reporter:
              Benoy Antony
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:

                Development