Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3304

[HOD] logcondense fails if DFS has files that are not log files, but match a certain pattern

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.16.2
    • 0.16.4
    • contrib/hod
    • None
    • Reviewed
    • Fixed the regular expression used by the logcondense utility to generate the list of log files to purge from HDFS. Also, improved error handling, so a single invalid log file does not fail the purge process.

    Description

      logcondense works by listing files in dfs and match them against a certain pattern. This pattern is incorrect in the sense that it can potentially match files that are not log files. And this can cause it to fail and not delete files correctly as it should.

      It should use a correct regular expression that will only list log files. Also, possibly it should log a stack trace if it happens, but continue to delete files and not stop.

      Attachments

        1. 3304.patch
          5 kB
          Hemanth Yamijala
        2. 3304.2.patch
          30 kB
          Hemanth Yamijala
        3. 3304.1.patch
          30 kB
          Hemanth Yamijala

        Activity

          People

            yhemanth Hemanth Yamijala
            yhemanth Hemanth Yamijala
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: