Details

    • Hadoop Flags:
      Reviewed

      Description

      Currently block location for writes are made based on:

      1. Datanode load (number of transceivers)
      2. Space left on datanode
      3. Topology

      With storage abstraction, namenode must choose a storage instead of a datanode for block placement. It also needs to consider storage type, load on the storage etc.

      As an additional benefit, currently HDFS support heterogeneous nodes (nodes with different number of spindles etc.) poorly. This work should help solve that issue as well.

        Attachments

        1. h4990_20130909.patch
          13 kB
          Tsz-wo Sze
        2. h4990_20130916.patch
          141 kB
          Tsz-wo Sze
        3. h4990_20130917.patch
          138 kB
          Tsz-wo Sze
        4. h4990_20130917b.patch
          137 kB
          Tsz-wo Sze
        5. h4990_20130917c.patch
          154 kB
          Tsz-wo Sze
        6. h4990_20130918.patch
          157 kB
          Tsz-wo Sze
        7. h4990_20130918b.patch
          157 kB
          Tsz-wo Sze

          Issue Links

            Activity

              People

              • Assignee:
                szetszwo Tsz-wo Sze
                Reporter:
                sureshms Suresh Srinivas
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: