Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3304

[HOD] logcondense fails if DFS has files that are not log files, but match a certain pattern

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.16.2
    • Fix Version/s: 0.16.4
    • Component/s: contrib/hod
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed the regular expression used by the logcondense utility to generate the list of log files to purge from HDFS. Also, improved error handling, so a single invalid log file does not fail the purge process.

      Description

      logcondense works by listing files in dfs and match them against a certain pattern. This pattern is incorrect in the sense that it can potentially match files that are not log files. And this can cause it to fail and not delete files correctly as it should.

      It should use a correct regular expression that will only list log files. Also, possibly it should log a stack trace if it happens, but continue to delete files and not stop.

        Attachments

        1. 3304.1.patch
          30 kB
          Hemanth Yamijala
        2. 3304.2.patch
          30 kB
          Hemanth Yamijala
        3. 3304.patch
          5 kB
          Hemanth Yamijala

          Activity

            People

            • Assignee:
              yhemanth Hemanth Yamijala
              Reporter:
              yhemanth Hemanth Yamijala
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: