Hadoop Common
  1. Hadoop Common
  2. HADOOP-4060

[HOD] Make HOD to roll log files on the client

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.1
    • Component/s: contrib/hod
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      HOD client was modified to roll over client logs being written to the cluster directory. A new configuration parameter, hod.log-rollover-count, was introduced to specify how many rollover log files to retain.

      Description

      Currently HOD writes a log file on the client in the cluster directory, named hod-<username>.log. This file is appended to for each run of hod allocation that runs on the same cluster directory. Thus, the file can become quite large - particularly if the job is queued for a long time. If there are problems with the HOD client subsequently, this could result in stale file handles pointing to this large file, and cause disk space to fill up. Another problem is that a large log file is usually unusable for a user.

      1. HOD-LOG-ROTATION.txt
        3 kB
        Vinod Kumar Vavilapalli
      2. HOD-LOG-ROTATION.1.txt
        3 kB
        Vinod Kumar Vavilapalli

        Activity

        Hemanth Yamijala created issue -
        Vinod Kumar Vavilapalli made changes -
        Field Original Value New Value
        Attachment HOD-LOG-ROTATION.txt [ 12389437 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment HOD-LOG-ROTATION.1.txt [ 12389551 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hemanth Yamijala made changes -
        Release Note HOD client was modified to roll over client logs being written to the cluster directory. A new configuration parameter, hod.log-rollover-count, was introduced to specify how many rollover log files to retain.
        Hadoop Flags [Incompatible change, Reviewed]
        Nigel Daley made changes -
        Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.19.0 [ 12313211 ]
        Resolution Fixed [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Hadoop Flags [Reviewed, Incompatible change] [Reviewed]
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development