Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4949

Centralized cache management in HDFS

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0, 3.0.0-alpha1
    • Fix Version/s: 2.3.0
    • Component/s: datanode, namenode
    • Labels:
      None

      Description

      HDFS currently has no support for managing or exposing in-memory caches at datanodes. This makes it harder for higher level application frameworks like Hive, Pig, and Impala to effectively use cluster memory, because they cannot explicitly cache important datasets or place their tasks for memory locality.

        Attachments

        1. caching-design-doc-2013-07-02.pdf
          270 kB
          Andrew Wang
        2. caching-design-doc-2013-08-09.pdf
          305 kB
          Andrew Wang
        3. caching-testplan.pdf
          99 kB
          Stephen Chu
        4. caching-design-doc-2013-10-24.pdf
          312 kB
          Colin P. McCabe
        5. HDFS-4949-consolidated.patch
          503 kB
          Andrew Wang
        6. hdfs-4949-branch-2.patch
          698 kB
          Andrew Wang

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                andrew.wang Andrew Wang
                Reporter:
                andrew.wang Andrew Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                104 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: