Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6584 Support Archival Storage
  3. HDFS-6969

Archival Storage: INode#getStoragePolicyID should always return the latest storage policy

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: balancer & mover, namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In general, every file should only provide exact one storage policy for the Mover, no matter its snapshot states. Suppose a file /foo/bar, and it is contained in snapshots s1 and s2 of the root. If /foo/bar, /.snapshot/s1/foo/bar and /.snapshot/s2/foo/bar have different storage policies, when running Mover, we have to select one of the storage policies, among which the latest one should be the best. And if /foo/bar is deleted, we should still use its storage policy before the deletion, since the file deletion should not trigger data migration.

      Thus maybe what we can do is:
      1. For a file with policy directly specified on it, alway follow the latest
      2. Otherwise follow its latest parental path to identify its storage policy (simply following the parent link)

        Attachments

        1. HDFS-6969.002.patch
          28 kB
          Jing Zhao
        2. HDFS-6969.001.patch
          28 kB
          Jing Zhao
        3. HDFS-6969.000.patch
          26 kB
          Jing Zhao

          Activity

            People

            • Assignee:
              jingzhao Jing Zhao
              Reporter:
              jingzhao Jing Zhao
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: