Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6919 Enforce a single limit for RAM disk usage and replicas cached via locking
  3. HDFS-8157

Writes to RAM DISK reserve locked memory for block files

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • datanode
    • None
    • Reviewed
    • This change requires setting the dfs.datanode.max.locked.memory configuration key to use the HDFS Lazy Persist feature. Its value limits the combined off-heap memory for blocks in RAM via caching and lazy persist writes.

    Description

      Per discussion on HDFS-6919, the first step is that writes to RAM disk will reserve locked memory via the FsDatasetCache.

      Attachments

        1. HDFS-8157.01.patch
          47 kB
          Arpit Agarwal
        2. HDFS-8157.02.patch
          45 kB
          Arpit Agarwal
        3. HDFS-8157.03.patch
          45 kB
          Arpit Agarwal
        4. HDFS-8157.04.patch
          49 kB
          Arpit Agarwal

        Issue Links

          is required by

          Sub-task - The sub-task of the issue HDFS-8192 Eviction should key off used locked memory instead of ram disk free space

          • Major - Major loss of function.
          • Resolved
          requires

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8152 Refactoring of lazy persist storage cases

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              arp Arpit Agarwal
              arp Arpit Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: