Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10593

MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • None
    • None
    • None
    • None

    Description

      In HDFS, "dfs.namenode.fs-limits.max-directory-items" was introduced in HDFS-6102 to restrict max items of single directory, and the value of it can not be larger than the value of MAX_DIR_ITEMS. Since "ipc.maximum.data.length" was added in HADOOP-9676 and documented in HADOOP-13039 to make maximum RPC buffer size configurable, it's not proper to hard code the value of MAX_DIR_ITEMS in FSDirectory.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6102 Lower the default maximum items per directory to fix PB fsimage loading

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13039 Add documentation for configuration property ipc.maximum.data.length for controlling maximum RPC message size.

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9676 make maximum RPC buffer size configurable

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              Unassigned Unassigned
              yuanbo Yuanbo Liu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: