Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11060

make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.3, 2.8.1, 3.0.0-alpha3
    • Fix Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2
    • Component/s: hdfs
    • Labels:
      None

      Description

      Current, the easiest way to determine which blocks is missing is using NN web UI or JMX. Unfortunately, because the DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED=100 is hard code in FSNamesystem, only 100 missing blocks can be return by UI and JMX. Even the result of URL "https://nn:50070/fsck?listcorruptfileblocks=1&path=%2F" is limited by this hard code value too.
      I did know FSCK can return more than 100 result but due to the security reason(with kerberos), it is very hard to involve to costumer programs and scripts.

      So I think it should add a configurable var "maxCorruptFileBlocksReturned" to fix above case.
      If community also think it's worth to do, I will patch this. If not, please feel free to tell me the reason.

        Attachments

        1. HDFS-11060.1.patch
          7 kB
          Lantao Jin
        2. HDFS-11060.2.patch
          4 kB
          Lantao Jin

          Activity

            People

            • Assignee:
              cltlfcjin Lantao Jin
              Reporter:
              cltlfcjin Lantao Jin
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: