Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.22.0
    • Fix Version/s: None
    • Component/s: tasktracker
    • Labels:
      None

      Description

      The tasktrackers cleanup the userlog directory when they restart.
      This happens independent of value of mapred.userlog.retain.hours.

      The feature is to add a configurable feature to respect mapred.userlog.retain.hours across TT restarts

        Issue Links

          Activity

          Hide
          qwertymaniac Harsh J added a comment -

          I wonder if MAPREDUCE-2415 broke this.

          It would get cleared even if a TT is sent a reinitialization signal from the JT too. We can perhaps selectively ignore "uesrlogs/" when deleting the local dirs and leave it to the userlog-manager to clean them up per the retention policy?

          Show
          qwertymaniac Harsh J added a comment - I wonder if MAPREDUCE-2415 broke this. It would get cleared even if a TT is sent a reinitialization signal from the JT too. We can perhaps selectively ignore "uesrlogs/" when deleting the local dirs and leave it to the userlog-manager to clean them up per the retention policy?
          Hide
          qwertymaniac Harsh J added a comment -

          No, MAPREDUCE-2415 should not have broken this:

             public void cleanupStorage() throws IOException {
          -    this.fConf.deleteLocalFiles();
          +    this.fConf.deleteLocalFiles(SUBDIR);
          +    this.fConf.deleteLocalFiles(TT_PRIVATE_DIR);
          +    this.fConf.deleteLocalFiles(TT_LOG_TMP_DIR);
             }
          

          As we can see there, it now deletes specific sub-dirs instead of all of them, when cleaning the local dirs.

          Show
          qwertymaniac Harsh J added a comment - No, MAPREDUCE-2415 should not have broken this: public void cleanupStorage() throws IOException { - this .fConf.deleteLocalFiles(); + this .fConf.deleteLocalFiles(SUBDIR); + this .fConf.deleteLocalFiles(TT_PRIVATE_DIR); + this .fConf.deleteLocalFiles(TT_LOG_TMP_DIR); } As we can see there, it now deletes specific sub-dirs instead of all of them, when cleaning the local dirs.
          Hide
          acmurthy Arun C Murthy added a comment -

          What is the use case?

          Show
          acmurthy Arun C Murthy added a comment - What is the use case?
          Hide
          qwertymaniac Harsh J added a comment -

          Arun - Isn't this a bug? Restarts of a service should not break the log retention guarantee?

          Show
          qwertymaniac Harsh J added a comment - Arun - Isn't this a bug? Restarts of a service should not break the log retention guarantee?
          Hide
          qwertymaniac Harsh J added a comment -

          I could not reproduce this on 1.0.3. I restarted the TT, re-initialized it (via JT restart), but the MR local directory userlogs/ still exist and continue to be symlinked inside the logs dir (and nowhere in code can I find a full clean delete of TaskLog.USERLOGS_DIR_NAME presently). Can you post some reproduce steps for this?

          Show
          qwertymaniac Harsh J added a comment - I could not reproduce this on 1.0.3. I restarted the TT, re-initialized it (via JT restart), but the MR local directory userlogs/ still exist and continue to be symlinked inside the logs dir (and nowhere in code can I find a full clean delete of TaskLog.USERLOGS_DIR_NAME presently). Can you post some reproduce steps for this?
          Hide
          benoyantony Benoy Antony added a comment -

          Sure, I will add the steps to reproduce.

          Show
          benoyantony Benoy Antony added a comment - Sure, I will add the steps to reproduce.
          Hide
          benoyantony Benoy Antony added a comment -

          This issue occurs only in those distributions where MAPREDUCE-2415 is applied as well as MRAsyncDiskService is used to cleanup the volumes during TT startup.
          This is not applicable to 1.0 or 1.1 since MRAsyncDiskService is not present in those.
          I don't think , it is applicable to trunk.

          Show
          benoyantony Benoy Antony added a comment - This issue occurs only in those distributions where MAPREDUCE-2415 is applied as well as MRAsyncDiskService is used to cleanup the volumes during TT startup. This is not applicable to 1.0 or 1.1 since MRAsyncDiskService is not present in those. I don't think , it is applicable to trunk.
          Hide
          qwertymaniac Harsh J added a comment -

          Thanks very much Benoy!

          Reopen reason: This should still affect 0.22 due to MAPREDUCE-1213 being in it. Lets try to fix it for 0.22.x.

          Show
          qwertymaniac Harsh J added a comment - Thanks very much Benoy! Reopen reason: This should still affect 0.22 due to MAPREDUCE-1213 being in it. Lets try to fix it for 0.22.x.
          Hide
          benoyantony Benoy Antony added a comment -

          This will not impact 0.22. MAPREDUCE -2415 was not ported to 0.22 . So userlogs directory will not be under the scratch directories.

          Show
          benoyantony Benoy Antony added a comment - This will not impact 0.22. MAPREDUCE -2415 was not ported to 0.22 . So userlogs directory will not be under the scratch directories.
          Hide
          qwertymaniac Harsh J added a comment -

          Ah yes, my bad. Would 0.22 benefit from MAPREDUCE-2415 though? If its not needed, we can probably close this out again.

          Show
          qwertymaniac Harsh J added a comment - Ah yes, my bad. Would 0.22 benefit from MAPREDUCE-2415 though? If its not needed, we can probably close this out again.
          Hide
          benoyantony Benoy Antony added a comment -

          Good point related to porting MAPREDUCE-2415 to 0.22.
          Another related question will be porting MAPREDUCE-1213 to 1.1

          Show
          benoyantony Benoy Antony added a comment - Good point related to porting MAPREDUCE-2415 to 0.22. Another related question will be porting MAPREDUCE-1213 to 1.1

            People

            • Assignee:
              Unassigned
              Reporter:
              benoyantony Benoy Antony
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development